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Environmental Protection Agency, through its Office of Research and Development, managed 
the research described herein. It has been subjected to the Agency’s peer and administrative 
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the author(s) and do not necessarily reflect the views of the Agency, therefore, no official 
endorsement should be inferred. Any mention of trade names or commercial products does not 
constitute endorsement or recommendation for use. 
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Foreword 


The U.S. Environmental Protection Agency (EPA) is charged by Congress with protecting the 
Nation’s land, air, and water resources. Under a mandate of national environmental laws, the 
Agency strives to formulate and implement actions leading to a compatible balance between 
human activities and the ability of natural systems to support and nurture life. To meet this 
mandate, EPA’s research program is providing data and technical support for solving 
environmental problems today and building a science knowledge base necessary to manage our 
ecological resources wisely, understand how pollutants affect our health, and prevent or reduce 
environmental risks in the future. 


The National Risk Management Research Laboratory (NRMRL) is the Agency’s center for 
investigation of technological and management approaches for preventing and reducing risks 
from pollution that threaten human health and the environment. The focus of the Laboratory’s 
research program is on methods and their cost-effectiveness for prevention and control of 
pollution to air, land, water, and subsurface resources; protection of water quality in public water 
systems; remediation of contaminated sites, sediments and groundwater; prevention and control 
of indoor air pollution; and restoration of ecosystems. NRMRL collaborates with both public 
and private sector partners to foster technologies that reduce the cost of compliance and to 
anticipate emerging problems. NRMRL’s research provides solutions to environmental 
problems by: developing and promoting technologies that protect and improve the environment; 
advancing scientific and engineering information to support regulatory and policy decisions; and 
providing the technical support and information transfer to ensure implementation of 
environmental regulations and strategies at the national, state, and community levels. 


This publication has been produced as part of the Laboratory’s strategic long-term research plan. 


It is published and made available by EPA’s Office of Research and Development to assist the 
user community and to link researchers with their clients. 


Sally Gutierrez, Director 
National Risk Management Research Laboratory 


ill 


Acknowledgments 


The authors wish to acknowledge the support of all those who helped plan and conduct the 
verification test, analyze the data, and prepare this report. We also would like to thank Al 
Liabastre, ret. U.S. Army Center for Health Promotion and Preventive Medicine; David Jacobs, 
National Center for Healthy Housing; Kenn White, American Industrial Hygiene Association; 
Larry Franklin, Centers for Disease Control and Prevention, Coordinating Center for 
Environmental Health and Injury Prevention, National Center for Environmental Health, Lead 
Poisoning Prevention Branch; and Moira Lataille and Michael Crane, U.S. EPA for their careful 
review of the test/quality assurance plan and this verification report. Quality assurance oversight 
was provided by Michelle Henderson, U.S. EPA, and Zachary Willenberg and Rosanna Buhl, 
Battelle. 


iv 


Contents 


Page 

INGE CE ok ecstatic Hane) celia Geben ternal Haga weed Gee ab aaecea tl eu dees pew date cadets aay sean li 
ORC WOPU, 3 Satete ates ews i Saas Seas hide Bl eee nd eee BAe eee ad BAe adule ili 
PRK NO WEG OTINE TIS Sasa att coi an Oca ct teed ee dah tiie coh cheba paar an ath Een e ok alee eta: iv 
List of Abbreviations ‘and A Cromyms:...c.ccssscesesssgevedcessnsecbes cavvaveestsvehensdasduassdeanted sabacsesseaseecetwetacss Vill 
Chapter]. back sro: se. scce.cccttet ccs usasccutlaes aecs gaa tu ceeuea pea eanau augue autadeamudaatde dameaeasde ties aeataataclaeees 1 
Chapter:2 “Technology IDESCrap tem a a5. caces coeds uecsoetscsdeccaslacsssesheasscedvesouavsane se qoos tasdee Syessnorneas eect Zz 
Chapter 3." lest Desi on and: Procedures iecies.cs.ccisas va scassesueadostaaecnde toaeaicoeamede sianadasacsuqniadadeeeaisnseeedeniaaees 4 
Se MOU IN, Bt See e seca S ak Sisceai Sat baat oekaki inl nt, ot oe cal ie IS cd Ameen it dees 4 
Ded ESC ACIIIGY can, cocoa say caverns masa taesaduaretaaresiia aso fesaleun unsere eta panacea ono aacala taeeas unucage ee aeuadeaneauoakuanaaees 5 
3,5, Pest. PIOcCdures: a Ae eee Ie Ce AO Ne RES Mees ia 5 
3.3.1 Test Sample Collection, Storage, and Shipment... eecceeeeeeeeeeeceeeeeeeteeeeeteeeees 8 

3.3.2 Test Sample Analysis Procedure oc.c6.is5sceissscvedave, jogecsausianes lesdadeadendecwsdecerstecsanmecasdeaes 8 
Chapter 4 Quality Assurance/Quality Control... eecceeesseceeseceececeeaaeeceeneeceeeeeceeeeecseeeeeeeeeees 12 
A We Oia lie COTO SARIS Sia 3a Sys ut sac as'gasy aos cede pce ecanauy pn cnvg uae daawes taaeae sxeanscsus bed aoevezdaceasesercs 12 
4.1.1 ICP-AES Blank Sample Results ijcwccccevegeiceesu ade he nue ee ene esac 12 

4.1.2 ICP-AES Matrix Spike Samples and Calibration Verification Standards.............. 12 

4.1.3 Test Kit Quality Controls and Blank PEMS.......... cscs cesceeeeeeeseeceaeceeeeseeeeaeeenaeenes 13 

BND EN MUOTNS oes Se enon ah apes nd SaSG Sts eeetesag leueceada ONG eee Redaa Sa eed On a acd ee ee Taos 14 
42 yh: Performance: B valuation Amis: c55 xsi as aes saad ects nso coda casero tas evans emo eae 14 

42.2 ‘Technical Systems: Audits c. cisczste6 pein cons Se gees devsanexadysgassdea Shes bas dnceseed gee) aoeedoceateagnneny 14 

472.3: Audit of Data Quality Audit s.ccccccasssccceasscajecesasepsdesisesdene sasadcaensededanepedaatansd cateunsoans 14 
Chapters /Statistieal Methods. sci ctia fae Site eset poate tendae dette en AQ aut een ie een ale 16 
DL Palsé.Positiverand Palse Ne Salve Rate saci oss cesyechs sad cess caevaaossudan os co3 sates qeaveastecoutancccnseetere 16 
D2 PIC CIS WOM 9900s a deaaeesenttsn ta tales ghee st Neunacitste dias Shara Paesleuadt os Gina ledtlaaSve uth de shea atlas areata walsand 17 
Fip3 DETISUULVILY cs sascaatanssdgucaswasde Dieasesannsaasanads shdacs shssausvas jdacesWuacad vs losdensaouande suundsanue suasaa ee destaccaneeeata ig 
5.4 Modeled Probability of Test Kit Response ............eceeeccecsscceceeececseececseececseeeesceeenaeeeeaeeees 17 
5.4.2 Accounting for Measurement Error — SIMEX Background and Intuition .............. 18 

5.4.3 SIMEX Input and Anal ysisis.cccicssciesiscevavsesavicvescsavesiced sdecacsnspavedvehevenccossaccesandevedeaaeasee 19 

5,44 SSOOUNESS OF Pils cohorsecfecx coma eraphaadenioctnaten yeast Goaunatgaetedl uatney cea Fet Goeibenaeateiqeayies 20 

Ds DL AUER ETC CUS ia ic a at ia ia cn tae edie Aa ed alec nd vase duelice laine 21 
DO Operational. PACtOrs.t.cccgsicescasscecstnctarseasessyaceaasiaseaveqaraetaat aaecases auqsey casa ceansorenetialanecssesseauaieaee 21 
Chapter6: Test ReESulits ieiiscacsteascacasonnsctnaraccdns Sedeaueedwabuceennes edbaronscateabeneadenvacgusepsadeuaunscdeateavdeenedeges 22 
6:1. False: Positive and: Palse Negative Rate sisvniveciiciigud tess dacesdetahenaeatoen teeancnaven gees sewesioesaesoqned dens D2 
G2 PTCCISION 055 ia oe th lie SR ee aie teh I ie a Ah OE Rah Bl aa 26 
G3” SETS Y Uy acess oz ons ooea cece xausads tessets Sa uincaaahayqaaesaeeacaes Goemece be pu cca ameter ccasis mht ee 29 
6.4: Modeled Probabilityof Test Kit. Responsess.4 cnjci pant nn ieaiesciedcamt aed oaneseletss 30 
G0 Matrix BItect ssisicc0 cua ite serlett teste toes aetna one ey eben aaee | 


6:6 Operational PActOrs ix, oe Gean fceaas ty lcy tansyaseavdahedeate becnsbeyahod aieatigcsageecton teas ducesuenyncgsuansoateueagoasaans 38 


Chapter] PerlormiitaniCe: Suminiary ;scincaaccascossccheetoncdacesaveeiaas eacenaveaneteiiaea ee ccananaeeaat oataueemeeaiies 40 

Chapter 0: RESTRIC ES: xs icc eese tec suc enced say ccs anedaiectadoa ans pnavuisas eas oes se asetnca ec nsec ueameaen vader aus sandeect pact 42 

Appendix A Performance Evaluation Materials Summary Information ..............eeeeeeseeeeeeees A-1 

Al Preparation of Performance Evaluation Materials 20.0.0... eseeeseceseceeeeeeeeceaecneeseeeeeneees A-3 
A2 Comparison of Expected vs. Actual Lead Concentrations of Performance Evaluation 

IMME TAS saeco eR Bd rset ta cs ea ac ea aa cael carseat ea A-19 

A3 QA/QC Results for the ICP-AES Analysis of Performance Evaluation Materials....... A-22 

Appendix BV endor C OmMents 5 ¢cceiescacsacionsso ness pcnkanfabesehe i QianaadoncsoeaiaeeGadad sAudetes tne aamaencoeen ees B-1 

Figures 
Bistre 2-1. The LeadPaint@ heck Test: Kit d.ev.qcssscsasesnscedevaseuinteccaasestodecascadeceud yeas zeaehoenamenghesenenseeenans 2 


Figure 6-1. Probability curves that represent test kit results that are both perfect (red line) and 
within RRP rule. criteria: (black Solid: 1iMe )s ican cncasseteddesdauedertinadeanosiareindes ood aca deinwncaehs 34 


Figure 6-2. Probability curves with shaded region to denote performance results that meet RRP 
tule false positive and negative criteria. Test kits with curves that fall within the white region 


and avoid the shaded region meet the RRP rule. 000.0... eee ceeeeeceeseeceseeeeceeeeeceeceeceeeeeceeeeceteeeesteeeees 35 

Figure 6-3. LeadPaintCheck test kit predicted probability of positive test result (solid lines) with 

90% prediction interval (dotted lines) for various substrates. ..........ecceeseeceeneeceeececeeeeeeeeeeeeeteeeees 36 
Tables 

Table 3-1. PEMs Testing Scheme for Each Test Kit..............ccesssccsscsssesrcessscesssseessssesenescessnceseenes 6 


Table 6-1. The number of panels in each false positive and false negative analysis category.... 23 


Table 6-2. LeadPaintCheck test kit false positive results for panels with confirmed lead levels < 
0.8 mg/cm? and false negative results for panels with confirmed lead levels = 1.2 mg/cm” Baie 24 


Table 6-3. LeadPaintCheck test kit false positive results for panels with confirmed lead levels < 
1 mg/cm’ and false negative results for panels with confirmed lead levels > 1 mg/cm’.............. 25 


Table 6-4. Actual lead levels and their replicate set labels ......... eee ceeceeceeseceeececeeeeeceeeeeceeeeeenaeees 26 
Table 6-5. The number of panels at each target level and the number in each replicate set bin . 27 


Table 6-6. LeadPaintCheck test kit consistency results by operator type, lead type, substrate, and 
HE eT (on col ireae ene ree me re DYE Dome eee PLONE ee Pee EDS et OODLE er ame eNO ENC err eee ae 28 


Table 6-7. LeadPaintCheck test kit precision results by lead type and operator type .................. 29 


Table 6-8. LeadPaintCheck test kit sensitivity results — lowest lead level for which the kit gave 
consistent positive results (Mg/CM) ......scsccscssssescecscssssssssssssessessssssessessssssssscsssssssssssseessssssessesessees 30 


Table 6-9. LeadPaintCheck test kit univariate associations between probability of positive 


response and Explanatory Variables .csccisisciccesavetene toe codassdesadavenaesevs vageca sacs sodas sdaden Slee geaatasvenaacs donde 30 
Table 6-10. LeadPaintCheck test kit multivariable Stata SIMEX logistic regression parameter 
CSUMALES Lips D ate aa ss date alaue heed ane aces rule aaa ata ealieta lee celts ha een ad gahetatiae 31 


Table 6-11. LeadPaintCheck test kit modeled probability of positive test results and upper 95% 
prediction bound when lead level = 0.8 Mg/CIM? o.....ceccsesessesssessesseessessessessesssessessesseessessesseeseesseanes 32 


Table 6-12. LeadPaintCheck test kit modeled probability of positive test results, lower 95% 
prediction bound, and corresponding conservative estimate of the false negative rate when lead 


level = 1.2 mg/cm? assdacuasuguensh qua zaeawesnatlagteotansaeaaieaeoeadeeay boowanseatiedeamesusoneal pip eaeetaesaanaeamone eee 52 


Table 6-13. LeadPaintCheck test kit false positive and negative threshold values (95% 
confidence) based on the modeled probability of test results 00.0... ee eeceeeseeeeseeeeeeteeeeneeeenaeeees 37 


vii 


Abs 
AMS 
ASTM 
CCV 
COC 
CRM 
EPA 
ESTE 
ETV 
HP 
ICP-AES 
LCS 
mg/em? 
uL 

mL 
MSDS 
NLLAP 
PE 
PEM 
ppb 

PT 

QA 

QC 
QCS 
QMP 
RRP 
SOP 
TSA 


List of Abbreviations and Acronyms 


absorbance 

Advanced Monitoring Systems 

American Society for Testing and Materials 
continuing calibration verification 

chain of custody 

certified reference material 

U.S. Environmental Protection Agency 
Environmental and Sustainable Technology Evaluations 
Environmental Technology Verification 
Homogenized Paint 

inductively coupled plasma-atomic emission spectrometry 
laboratory control spike 

milligrams per centimeter squared 

microliter 

milliliter 

material safety data sheets 

National Lead Laboratory Accreditation Program 
performance evaluation 

performance evaluation material 

parts per billion 

performance test 

quality assurance 

quality control 

quality control sample 

quality management plan 

Renovation, Repair, and Painting 

standard operating procedure 

technical systems audit 


Vill 


Chapter 1 
Background 


The U.S. Environmental Protection Agency (EPA) supports the Environmental Technology 
Verification (ETV) Program to facilitate the deployment of innovative environmental 
technologies through performance verification and dissemination of information. The goal of the 
ETV Program is to further environmental protection by accelerating the acceptance and use of 
improved and cost-effective technologies. ETV seeks to achieve this goal by providing high- 
quality, peer-reviewed data on technology performance to those involved in the design, 
distribution, financing, permitting, purchase, and use of environmental technologies. 


ETV works in partnership with recognized testing organizations; with stakeholder groups 
consisting of buyers, vendor organizations, and permitters; and with the full participation of 
individual technology developers. The program evaluates the performance of innovative 
technologies by developing test plans that are responsive to the needs of stakeholders, 
conducting field or laboratory tests (as appropriate), collecting and analyzing data, and preparing 
peer-reviewed reports. All evaluations are conducted according to rigorous quality assurance 
(QA) protocols to ensure that data of known and adequate quality are generated and that the 
results are defensible. 


This verification test was conducted under the U.S. EPA ETV program. Testing was performed 
by Battelle, which served as the verification organization under the Environmental and 
Sustainable Technology Evaluations (ESTE) arm of ETV. Battelle evaluated the performance of 
qualitative spot test kits for lead in paint. 


This verification test was developed with the support of a stakeholder technical panel. A 
voluntary stakeholder technical panel consisting of individuals from the American Industrial 
Hygiene Association (Kenn White), U.S. Department of Housing and Urban Development 
(Warren Friedman), National Institute for Occupational Safety and Health (Kevin Ashley), U.S. 
Army Center for Health Promotion and Preventative Medicine (Al Liabastre), National Center 
for Healthy Housing (David Jacobs), National Association of Homebuilders (Matt Watkins), the 
U.S. Consumer Product Safety Commission (Joanna Matheson), the Center for Disease Control 
and Prevention (Larry Franklin), and U.S. EPA (Paul Carroll and Moira Lataille) was formed for 
this verification test. Participants on this panel were reviewed and approved by EPA. This panel 
gave input during the entire ETV process, including providing guidance and input on the 
development of the performance evaluation materials used in this test, on the development of the 
test design and test/QA plan, and comments on this report. 


Chapter 2 
Technology Description 


This report provides results for the verification testing of LeadPaintCheck test kit for lead-based 
paint by Industrial Test Systems, Inc. The following is a description of the LeadPaintCheck test 
kit, based on information provided by the vendor. The information provided below was not 
verified in this test. 


LeadPaintCheck is a semi-quantitative test kit that can detect the presence or absence of lead in 
paint. To evaluate a sample, a half-inch circle paint sample is collected, the paint sample is cut 
up in smaller pieces, and then it is homogenized in the ACID-1 acid solution. Lead in the 
Homogenized Paint (HP) sample is solubilized as Pb*2 by the acid. The HP sample is then 
diluted 1/500 (100 microliters (ul) per 50 milliliters (mL) of water). 


The procedure for measurement of lead in the diluted sample uses 4 mL of the HP extract, which 
are added to the built-in photocell of the eXact® LEADQuick™ Photometer. The sample is 
zeroed, two drops of Pb-2 Buffer 
are added to make the solution 
mildly alkali (pH 9-10), and an 
eXact® Strip Pb-3 is then dipped 
in the sample for 20 seconds with 
gentle motion. The PB-3 strip 
simultaneously adds a porphyrin 
colorimetric indicator and mixes 
the solution. After one minute, in 
which time the porphyrin and Pb*2 
forms a colorimetric complex, the 
eXact® LEADQuick™ 
Photometer measures the optical 
density (color) as an absorbance 
number (abs). The “abs” value 
will be dependent on the amount of 
lead that is present: the higher the 
“abs” value, the higher the lead 
concentration in the sample. 





Figure 2-1. The LeadPaintCheck Test Kit 


The test kit determines when lead levels are below 1.0 mg/cm”. The test also identifies semi- 
quantitatively if the lead concentrations in the paint are 1, 1.5, 2, or 3 (and above) mg/cm’. 


The Startup Kit contains the waterproof eXact® LEADQuick™ photometer and enough reagents 
for 100 tests. Sample collection and preparation components include an Omni Homogenizer kit 
with stand, 15ml and 50ml conical tubes with rack, a 500ml wash bottle, 2” and 4’ cork bores 
with a sharpener and cork bore core remover, a 100ul pipette with pipette tips, scissors, forceps, 
a funnel system and small brush, a scalpel with blades, masking tape, an instructional video CD 
and a LeadPaintCheck standard. 


The LeadPaintCheck Startup Kit is priced at $1999.99. The reagent replacement set is $99.99 
for 50 tests. After the initial Startup investment the cost per paint test is $2. The estimated time 
to run the test, which includes the sampling, homogenizing, diluting, and testing time, is under 
15 minutes per each paint sample. 


Chapter 3 
Test Design and Procedures 


3.1 Introduction 


This verification test was conducted according to procedures specified in the Test/QA Plan for 
Verification of Qualitative Spot Test Kits for Lead in Paint.' Lead-based paints were commonly 
used in houses in both interior and exterior applications prior to 1978, when the US government 
banned the use of lead-based paint in residential applications. The term lead-based paint means 
paint or other surface coatings that contain lead at contents that equal or exceed a level of 1.0 
milligrams per centimeter squared (mg/cm) or 0.5 percent by weight.” This paint still exists in 
many of these houses across the country. The accurate and efficient identification of lead-based 
paint in housing is important to the Federal government as well as private individuals living in 
residences containing such paints. Renovation, repair, and painting (RRP) activities may disturb 
painted surfaces and produce a lead exposure hazard. Such disturbances can be especially 
harmful to children and pregnant women as lead exposure can cause neurological and 
developmental problems in both children and fetuses. In fact, because of the large amount of 
pre-1978 housing stock, a report by the President’s Task Force on Environmental Health Risks 
and Safety Risks to Children found that approximately 24 million US dwellings were at risk for 
lead-based paint hazards.” 


There are lead-based paint test kits available to help home owners and contractors identify lead- 
based paint hazards before any RRP activities take place so that proper health and safety 
measures can be taken. However, many of these test kits have been found to have high rates of 
false positives (i.e., test kit indicates that lead in excess of 1.0 mg/cm” is present, while in fact 
the true lead level is below 1.0 mg/em’).* This verification test was conducted in response to the 
call of the Renovation, Repair, and Painting rule’ for an EPA evaluation and recognition program 
for test kits that are candidates to meet the goal of a demonstrated probability (with 95% 
confidence) of a false negative response less than or equal to 5% of the time for paint containing 
lead at or above the regulated level, 1.0 mg/cm” and a demonstrated probability (with 95% 
confidence) of a false positive response less than or equal to 10% of the time for paint containing 
lead below the regulated level, 1.0 mg/cm’. This test incorporated ASTM International’ s E1828, 
Standard Practice for Evaluating the Performance Characteristics of Qualitative Chemical Spot 
Test Kits for Lead in Paint? guidelines into the test design. 


The objective of this verification test was to evaluate the performance of the test kits for the 
detection of lead in paint. This evaluation assessed the capabilities of the lead paint spot test kits 
against laboratory-prepared performance evaluation material (PEM) samples and compared the 


lead paint test kit results with those of a standard technique, inductively coupled plasma-atomic 
emission spectrometry (ICP-AES). Additionally, this verification test relied on verification 
testing staff observations to assess other performance characteristics of the lead paint test kits. 
Only qualitative results (e.g., detect/non-detect of lead at specified levels) were considered for 
each technology. 


The LeadPaintCheck test kit was verified by evaluating the following parameters: 


e False positive and false negative rates 

e Precision 

e Sensitivity 

e Modeled probability of test kit response 
e Matrix effects 

e Operational factors. 


Verification testing of the test kit was conducted from January to June 2010. This timeframe 
included testing of the test kit and also completion of all ICP-AES and QC analyses. False 
positive and negative rates were determined by comparing test kit responses to actual lead 
concentrations of the PEM as determined through ICP-AES. Precision was determined by 
reproducibility of responses for replicate samples. Sensitivity was determined as the lowest 
detectable level of the test kit. The modeled probability and matrix effects were determined 
using logistic regression models. 


Operational factors such as ease of use, operator bias, average cost, average time for kit 
operation, helpfulness of manuals, and sustainability metrics such as volume and type of waste 
generated from the use of each test kit, toxicity of the chemicals used, and energy consumption 
were determined based on documented observations of the testing staff and the Battelle 
Verification Test Coordinator. Operational factors were described qualitatively, not 
quantitatively; therefore, no statistical approaches were applied to the operational factors. 


3.2 Test Facility 


Laboratory analyses of the LeadPaintCheck test kit were conducted in Battelle laboratories in 
Columbus, Ohio. No field testing was conducted during this technology verification. 


3.3 Test Procedures 


Qualitative spot test kits for lead in paint were evaluated against a range of lead concentrations in 
paint on various substrates through the use of PEMs. PEMs were 3 inch by 3 inch square panels 
of wood (pine and poplar), metal, drywall, or plaster that were prepared by Battelle.° Pine and 
poplar were chosen for the wood panels as they are representative of woods most commonly 
found in homes. Table 3-1 shows the PEMs prepared for each test kit. Poplar and pine PEMs 
were distributed in random mixtures (e.g., two poplar and one pine or one poplar and two pine) 
for each set of three wood PEMs listed in Table 3-1. Each PEM was coated with either white 
lead (lead carbonate) or yellow lead (lead chromate) paint. The paint contained lead targeted at 


Table 3-1. PEMs Testing Scheme for Each Test Kit? 








Lead Type 


Control Blank 


Lead Level 
(mg/cm’) 


Substrate 
Wood 


PEMs Analyzed Per Test Kit by Topcoat Color 


Red-Orange 
3 3 


Grey-Black 


3 


Total 


9 





Metal 





Drywall 





Plaster 





White Lead 
(Lead Carbonate) 


Wood 





0.3 


Metal 





Drywall 
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Wood 





Metal 
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Drywall 





Plaster 





Wood 
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Metal 
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Metal 
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6.0 
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Yellow Lead 
(Lead Chromate) 


Wood 





Metal 





0.3 
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Plaster 





Wood 





0.6 


Metal 





Drywall 





Plaster 
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1.0 
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Plaster 





Wood 
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6.0 
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Unpainted PEMs Subtotal (2 per each substrate) 
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Total 








476 








* Actual number of PEMs used to evaluate performance at specific lead levels varied based on actual concentrations observed during analysis. 








0.3, 0.6, 1.0, 1.4, 2.0, and 6.0 mg/cm’. These lead concentrations were chosen with input from 
the stakeholder technical panel based on criteria provided in EPA’s lead RRP rule as well as to 
represent potential lead levels in homes. Paint containing no lead (0.0 mg/cm’) was also applied 
to each substrate and tested. 


Two different layers of paint were applied over the leaded paint. One was a primer designed for 
adhesion to linseed oil-based paint and the second coat was a typical interior modern latex paint 
tinted to one of three colors: white, red-orange, or grey-black. These colors were chosen by 
EPA, with input from the stakeholder technical panel, based on the potential of certain colors to 
interfere or not with lead paint test kit operations. The topcoat paint manufacturers’ 
recommended application thickness was used. Two coats at the recommended thickness were 
applied. Details on the PEM production process can be found in Appendix A. 


The LeadPaintCheck test kit for lead paint was operated by a technical and non-technical 
operator. The technical operator was a Battelle staff member with laboratory experience. The 
technical operator was trained by a representative of the vendor company in the operation of its 
test kit. The same technical operator operated this test kit throughout testing. Because this lead 
paint test kit is anticipated to be used by certified remodelers, renovators, and painters, it was 
also evaluated by a non-technical operator. The non-technical operator was a certified renovator 
with little to no experience with lead analysis. The non-technical operator was provided the 
instruction manual, demonstrational DVD, and other materials typically provided by the vendor 
with the test kit for training. The non-technical operator viewed the materials himself to 
understand how to operate the test kit. The non-technical operator was also permitted to ask 
questions or clarifications of the vendor on the operation of the test kit. This scenario 
approximated the training renovators are expected to receive under the RRP rule. 


Tests were performed in duplicate on each PEM by each operator, technical and non-technical 
(i.e., two samples were taken from each PEM by each operator). Duplicates were tested in 
succession by each operator on a given PEM. PEMs were analyzed blindly by each operator in 
that the PEMs used for analysis were marked with a non-identifying number. Test kit operators 
were not made aware of the paint type, lead level, or substrate of the PEM being tested. PEMs 
were tested in random order (i.e., PEMs were placed in plastic bins and the operators arbitrarily 
selected a PEM for analysis). To determine whether the substrate material affected the 
performance of the test kits, two unpainted PEMs of each substrate were tested using each test 
kit, in the same manner as all other PEMs (i.e., per the test kit instructions). Three PEMs at each 
lead level, substrate, and topcoat color were prepared for use in this test. In total, 468 painted 
PEMs were prepared for use in the verification test of each test kit. 


Paint chip samples from each PEM were analyzed by a National Lead Laboratory Accreditation 
Program (NLLAP) recognized laboratory, Schneider Laboratories, Inc., using ICP-AES to 
confirm the lead level of each PEM used for testing. The paint chip samples for reference 
analyses were collected by Battelle according to a Battelle SOP’, which was based on ASTM 
E1729.° The reference analyses confirmed the lead level of each PEM. Lead levels determined 
through the reference analysis were used for reporting and statistical analyses. 


The procedures for collecting, storing, and shipping test samples are provided below. 


3.3.1 Test Sample Collection, Storage, and Shipment 


Chips of lead paint were taken from each PEM and sent for ICP-AES analysis at a NULAP- 
recognized laboratory, Schneider Laboratories, Inc. A glass screw-top vial was labeled with the 
PEM identification number located on the back of the panel. The number was also recorded on 
the Chain of Custody (COC) form. Sampling was performed per the Battelle SOP for collection 
of dried paint samples for lead determination. All safety precautions and personal protective 
equipment were used. A one inch square, metal template was placed adjacent to the tested area. 
A utility knife was used to trace around the template. Tweezers and a utility knife were used to 
scrape and remove the paint within the one inch area, using caution to minimize introduction of 
the substrate into the paint sample. The topcoat and remaining paint were transferred to a 
glassine weighing paper with the assistance of a paintbrush. The sample was then transferred 
from the glassine paper into a glass vial using the paintbrush. All instruments and templates were 
wiped with tissue paper and the bench top was cleaned and gloves were changed between each 
sample to minimize contamination. The paint brush was carefully flicked and tapped over a trash 
can to remove any residual lead dust. All wipes and gloves were disposed of as lead waste. The 
vials were then collected into a zip-top bag and taped up securely for shipping. The bags and 
COC were then shipped together using overnight delivery to Schneider Laboratories, Inc. 


Paint chip samples were stored at room temperature as received by Schneider Laboratories, Inc. 
and then analyzed by ICP-AES. Analytical results were reported to Battelle within 2-3 days. 
Sample digests were stored separately by Schneider Laboratories, Inc. at room temperature. 


PEMs were stored individually in zip-top bags. The back of each PEM was labeled with an 
identifying number. The outside of the zip-top bag was labeled with the same number. Each 
PEM was wrapped in a Kimwipe and each zip-top bag was sealed when not in use. The zip-top 
bags containing the PEMs were housed in large plastic bins in the laboratory during testing. 


3.3.2 Test Sample Analysis Procedure 


The LeadPaintCheck test kit contained all of the components necessary to collect and analyze a 
paint sample, except white print paper and water. 


Paint samples were obtained from the selected PEMs by one of two methods, depending on the 
type of substrate. For the drywall, wood, and plaster substrates, a ¥2 inch brass cork borer tool 
was used. A twisting motion while applying moderate pressure cut the sample from the PEM. 
The cork borer tool had to be sharpened before each use using the cork borer sharpening tool 
provided with the test kit. For the metal substrate, an X-acto” knife was used. The 
LeadPaintCheck test kit instructions call for using a ruler to measure a ¥2 inch x 2 inch area on 
metal substrates for removal of a paint sample. Because of the large number of samples being 
generated in this study, a stainless steel template was used by both the technical and non- 
technical operator. The template was traced around with an X-acto® knife. All paint samples 
were collected with the PEM placed on a clean sheet of white printer paper. A new sheet of 
paper was used for each PEM. 


Once the paint samples were removed from the PEM, they were cut into small pieces using 
either scissors or the provided X-acto® knife on a clean sheet of white printing paper. A 15 mL 


conical tube was labeled with the PEM identification and then screwed onto the end of the 
provided funnel system. The funnel was then hung vertically on wood boards. The cut paint 
samples were then transferred from the paper into the funnel system. A brush was used to sweep 
all of the paint material from the funnel into the 15 mL tube. The sides of the funnel were tapped 
to dislodge any stuck paint chip samples. 


Much of the testing for the LeadPaintCheck was conducted during cold, dry weather. As such, 
static electricity problems were encountered when transferring the paint chip samples into the 
funnel and subsequently into the attached 15 mL tube. The paint sample would cling to the 
plastic funnel which made it difficult to transfer the entire sample into the 15 mL tube using the 
funnel. Based on vendor-provided recommendations, the funnel was sprayed with Static Guard® 
at the beginning of each testing day to remove the static problem. The funnel was allowed to dry 
after being sprayed before it was used to collect a sample. The funnel was wiped out between 
samples. All instruments used to collect paint chips were also wiped between samples to 
eliminate any chance of cross-contamination. The paint brush was carefully flicked and tapped 
over a trash can to remove any residual dust. 


As noted above, the LeadPaintCheck instructions call for the use of the provided funnel system 
to collect a paint chip sample. In collecting paint samples from a vertical surface, the funnel was 
taped to the wall using masking tape and the paint chip sample was collected from that wall over 
the funnel. Because the PEMs used in this test were stand-alone pieces of substrate, the PEMs 
were placed on a piece of paper to collect the sample, but a device was prepared to allow the 
funnel system to be hung vertically to wood boards to simulate the use of the funnel to transfer 
the cut paint sample in a real-world environment. 


Once the sample had been transferred through the funnel to the graduated conical 15 mL test 
tube, the tube was removed from the funnel. Then 2 mL of the eXact® Reagent Acid-1 were 
added to the tube using the supplied dropper bottle. The sample was then ready to be 
homogenized. 


Before the sample could be homogenized, the homogenizer had to be assembled. The 
homogenizer unit was affixed to a ring stand using the supplied holding device. The stainless 
steel tip holding assembly was then attached to the unit by tightening the lock screw. A clean 
Omni Tip was attached by pushing up into the holding assembly and twisting clockwise until it 
stopped. 


After the homogenizer was assembled, the sample (in the 15 mL sample tube) was positioned 
such that the Omni Tip was all the way to the bottom of the tube. The homogenizer was set on 
maximum revolutions per minute and then turned on. During this digestion step, the sample tube 
was moved up and down about | inch, with a twisting motion, every 5 seconds to ensure all the 
paint chips were being ground. This homogenization was conducted for two minutes for each 
sample. At the end of the two minutes, with the homogenizer off, the sample tube was partially 
withdrawn from the Omni Tip and the Omni Tip was rinsed with deionized (DI) water to 
dislodge any paint particles. 


After each use, the Omni Tip was cleaned thoroughly with tap water and a brush to ensure that 
all remnants of the previous paint sample were removed, and then rinsed with DI water and dried 
before re-use. The LeadPaintCheck instructions only call for rinsing the Omni Tip in tap water 
after each use. Additional cleaning was conducted for this test to avoid any potential cross- 
contamination, though the vendor indicated that this was precautionary since just rinsing should 
be sufficient. The Omni Tips were discarded after 20 mixings and replaced with a new tip. 


The sample tube was then placed in a test tube rack and filled to the 15 mL line with DI water 
using the supplied squirt bottle. In the event of foaming during the digestion process, the sample 
tube was set aside until the foam dissipated before adding the DI water to the 15 mL mark. The 
sample tube was then capped and mixed by shaking four or five times. This is the homogenized 
paint (HP) sample. 


After the HP sample was ready, a 1:500 dilution tube was prepared by adding 50 mL DI water to 
a 50 mL conical, graduated tube. Using a 100 uL pipette, 100 uL of the HP sample were 
extracted and dispensed into the 50 mL dilution tube. The dilution tube was capped and then 
mixed by shaking four or five times. This was the diluted HP sample. This dilution was then 
read within two hours of preparation. All HP samples and dilutions were prepared and analyzed 
in the same day. 


Next, a quality control check was performed using the eXact® LEADQuick™ Photometer and a 
prepared standard. The photometer was turned on by pressing the Zero/On button. Then the 
Menu button was pressed and then re-pressed until PAI was displayed. The receptacle cell in the 
photometer was rinsed three times with DI water using the supplied squirt bottle. Then the 
receptacle cell in the photometer was rinsed three times with the calibration standard. Finally, the 
receptacle cell was filled to capacity (4 mL) with the calibration standard. Two drops of the 
eXact® Reagent Pb-2 were added to the cell. Next the eXact® Strip Pb-3 was dipped into the cell 
and the Read button was immediately pressed. The 20 second countdown on the timer was now 
activated. During this time, the eXact® Strip Pb-3 was gently moved back and forth to release 
the indicator agent. The strip was removed and discarded after the “1” on the display meter 
disappeared. 


The meter then read the sample for the next 60 seconds using a count up display. At the end of 
the cycle, an absorbance (abs) value was displayed. An abs value of 0.320 to 0.370 indicated the 
meter was calibrated correctly. If the value was outside of this range, the dilution and calibration 
steps were repeated. The photometer cell was cleaned by pouring out sample into the sink and 
rinsing it three times with DI water. The photometer was calibrated at the beginning of each test 
period. 


Once the eXact® LEADQuick™ Photometer was calibrated, the sample could be analyzed. To 
analyze the paint samples using the eXact® LEADQuick™ Photometer, the steps described 
above for evaluating the calibration standard were followed. Instead of rinsing and adding the 
calibration standard, the diluted HP sample was used and added. The abs reading for the sample 
was then compared to a supplied table in the test kit instructions to determine the lead content of 
the sample. Results were then reported as Negative (lead below 1.0 mg/cm’), Borderline, 
Positive for Lead (lead at 1.0 mg/cm”), Positive for Lead (lead at about 1.5 mg/cm’) and High 


Positive for Lead (at about 2.0 or 3.0 mg/cm’). Results that were Borderline were re-run starting 
with the 50 mL dilution step. 


Chapter 4 
Quality Assurance/Quality Control 


QA/QC procedures were performed according to the quality management plan (QMP) for the 
Battelle ETV Advanced Monitoring Systems (AMS) Center’, except where differences were 
noted for ESTE per the EPA ETV Program QMP””, and the test/QA plan for this verification 
test. Test procedures were as stated in the test/QA plan; however a deviation to the test/QA plan 
was made during the ICP-AES analyses. For some sample runs, continuous calibration 
verification (CCV) samples were run once every 20 instead of 10 samples. This deviation is 
described below. This change was assessed to have no impact on the quality of the results as 
described below. QA/QC procedures and results are described below. Additional information 
on QA/QC outcomes for the PEMs are provided in Appendix A. 


4.1 Quality Control Samples 


Steps were taken to maintain the quality of data collected during this verification test. This 
included analyzing specific quality control samples for the reference method (ICP-AES) and the 
test kit. 


4.1.1 ICP-AES Blank Sample Results 


Various blank samples were analyzed for the ICP-AES analyses. Method blank samples were 
analyzed in each set of 10-20 paint samples to ensure that no sources of contamination were 
present. An initial calibration blank was analyzed at the beginning of each run and used for 
initial calibration and zeroing the instrument. A continuing calibration blank was analyzed after 
each CCV to verify blank response and freedom from carryover. No blank samples failed during 
the analyses. 


4.1.2 ICP-AES Matrix Spike Samples and Calibration Verification Standards 


Initial calibration standards were run at the beginning of each set of analyses. The acceptance 
criterion for the calibration coefficient of the calibration standards was > 0.998. If this criterion 
was not met, the analysis was stopped and recalibration was performed before samples were 
analyzed. A 500 parts per billion (ppb) CCV standard was analyzed at the beginning of each run 
(following the initial calibration), at the end of each run, and every 10-20 samples. CCV 
recoveries ranged from 96% to 108%. Per the test/QA plan’, CCV sample frequency was once 
every 10 samples. For most of the sample sets CCVs were performed with this frequency. 


However, for later sample sets CCVs were run once every 20 samples. CCV samples were used 
to verify instrument performance. CCV samples were run every 10 samples as a preventative 
measure so that large amounts of samples do not need to be re-run if a CCV sample fails. In the 
course of this study, one CCV sample failed. All samples from the last passing CCV of that 
sample set were re-analyzed. 


A matrix spike sample and laboratory control sample (LCS), as well as duplicates of these 
samples, were also analyzed. Duplicate samples were run once every 10-20 samples. Acceptable 
recoveries for matrix spike samples were between 80-120%. Acceptable recoveries for LCS 
samples were between 80-120%. Duplicate samples had acceptance criteria of +25% relative 
percent difference (RPD). 


All matrix spike samples were performed as post-digestion spikes as there was insufficient 
sample volume to perform a pre-digestion spike. Matrix spike recoveries ranged from 86% to 
207%. Six matrix spike samples failed, with recoveries above the specified acceptance criteria. 
In these instances, the lead concentration in the sample was well above the spike level. Matrix 
spike results indicated that matrix interferences were not observed. Duplicate samples were 
within the specified RPD. 


LCS samples were analyzed once every 10-20 samples. LCS recoveries ranged from 17% to 
225%. Schneider Laboratories, Inc. noted that LCS failures on one sample set were attributed to 
improper spiking technique. Training on spiking procedures was immediately implemented by 
Schneider Laboratories for all analysts spiking samples. All LCS failures occurred prior to a 
revision to the Schneider Laboratories, Inc. SOP" for analyzing paint samples written 
specifically for this verification test. In the original version of the SOP, LCS samples were 
prepared by spiking a known amount of lead onto a certified reference material (CRM). This 
practice was changed because there were over-recovery issues. This was because the spike was 
not >3x the background lead concentration because of the high lead concentrations in the actual 
CRM samples. In the revised SOP, the LCS was prepared by spiking a piece of lead-free latex 
paint. There were no LCS failures after that. In addition, a QC check sample containing only 
the CRM, which had a known concentration of lead weighed out to a particular amount, was 
analyzed with each sample set throughout the verification test. These QC check samples all 
passed acceptance criteria. 


4.1.3 Test Kit Quality Controls and Blank PEMs 


As indicated in Section 3.3.2, specific quality control measures were part of the operation of the 
test kit itself. A quality control check was performed using the eXact® LEADQuick™ 
Photometer and a prepared standard. This quality control check was designed to confirm that 
pipetting and testing techniques are being properly implemented for the test kit. The quality 
control check standard results absorbance reading had to fall within a specific range to verify that 
the equipment and technique used are within expected limits. No quality control checks failed 
during testing. 


Painted PEMs containing no lead as well as each of the PEM substrates containing no paint were 
also run as part of the verification test. All samples of PEM substrates containing no paint 


returned negative results from the test kit (i.e., no lead was present). All samples of painted 
PEMs containing no lead returned negative results. 


4.2 Audits 


Three types of audits were performed during the verification test: a performance evaluation (PE) 
audit of the reference method measurements made in this verification test, a technical systems 
audit (TSA) of the verification test performance, and a data quality audit. Audit procedures are 
described below. 


4.2.1 Performance Evaluation Audits 


A PE audit was conducted to assess the quality of the reference method measurements made in 
this verification test. The reference method PE audit was performed by supplying an 
independent, NIST-traceable lead paint standard (Reference Material 8680, panel CB3), to the 
reference laboratory. The PE audit samples were analyzed in the same manner as all other 
samples and the analytical results for the PE audit samples were compared with the nominal 
concentration. The target criterion for this PE audit was in agreement with the analytical result 
within 20% of the nominal concentration. The specified acceptable concentration range for the 
NIST standard panel was 1.13 — 1.75 mg/cm’ (1.44 +0.31 mg/cm’). The PE samples taken from 
this standard panel were 1.38, 1.38, 1.19, and 1.31 mg/cm’. The PE audit result met the target 
criterion. This audit was performed once at the start of the test. 


4.2.2 Technical Systems Audit 


The Battelle Quality Manager performed one TSA during this verification test to ensure that the 
verification test was being performed according to the Battelle AMS Center and ETV Program 
QMPs, the test/QA plan, any published reference methods, and standard operating procedures. In 
the TSA, the Battelle Quality Manager reviewed the reference methods used, compared actual 
test procedures with those specified or referenced in the test/QA plan, and reviewed data 
acquisition and handling procedures. Also in the TSA, the Battelle Quality Manager observed 
testing, observed reference method sample preparation and analysis, inspected documentation, 
and reviewed technology-specific record books. He also checked standard certifications and 
technology data acquisition procedures and conferred with the technical staff. A TSA report was 
prepared. There were no findings. The records concerning the TSA are permanently stored with 
the Battelle Quality Manager. 


The EPA ETV Quality Manager also performed a TSA of both the reference laboratory and the 
testing conducted at Battelle Columbus, OH facilities. No findings were reported in the TSA of 
the reference laboratory, Schneider Laboratories, Inc. In the TSA of the lead paint test kit 
evaluations at Battelle’s Columbus, OH facilities, the EPA ETV Quality Manger cited two 
findings. These findings were related to ease of use observations and were immediately and 
adequately addressed and did not affect the quality of the test. 


4.2.3 Audit of Data Quality Audit 


Records generated in the verification test received a one-over-one review (i.e., review by a 


Battelle technical staff who did not generate the records) before these records were used to 
calculate, evaluate, or report verification results. A Battelle technical staff member involved in 
the verification test reviewed the data. Datasheets generated by the operators during testing were 
reviewed for completeness and errors. The person performing the review added his/her initials 
and the date to a hard copy of the record being reviewed. At least 10% of the data acquired 
during the verification test, including the ICP-AES results, were audited by Battelle. At least 
25% of the ICP-AES data acquired during the verification test were audited by EPA. Battelle’s 
Quality Manager traced the data from the initial acquisition, through reduction and statistical 
analysis, to final reporting to ensure the integrity of the reported results. All calculations 
performed on the data undergoing the audit were checked. Minor transcription errors were 
identified and corrected before the results were used for the calculations described in Chapter 5. 
Battelle’s and EPA’s Quality Managers also reviewed the PEM ICP-AES results thoroughly to 
ensure that all data quality indicators as stated in the test/QA plan were followed and that 
reported results matched the data generated on the instrument. Findings were cited by the EPA 
Quality Manager. Appropriate corrective actions were taken. Significant QA/QC concerns 
identified during EPA’s audit are discussed in Section 4.1. 


Chapter 5 
Statistical Methods 


The statistical methods used to evaluate the performance factors listed in Section 3.1 are 
presented in this chapter. The LeadPaintCheck test kit was evaluated for qualitative results (i.e., 
positive/negative responses to samples). All data analyses were based on these qualitative 
results. QC samples and unpainted PEM substrates were not included in any of these analyses. 
Results are provided in Chapter 6. 


5.1 False Positive and False Negative Rates 


A false positive response was defined as a positive result when regulated lead-based paint was 
not present. The test/QA plan’ defined false positive rates as being based on target lead levels at 
and below 0.6 mg/cm’ with confirmed values not to exceed 0.8 mg/cm’. Because confirmed lead 
levels of particular PEMs did not sometimes match target concentrations for those PEMs, false 
positive rates were assessed on panels with confirmed lead levels at 0.8 mg/cm? and lower. 
Consistent with the EPA’s April 22, 2008 RRP rule’, panels with an ICP-AES confirmed lead 
level between 0.8 and 1.0 mg/cm’ were not used in the false positive analysis. 


A false negative response was defined as a negative response when regulated lead-based paint 
was present. The test/QA plan defined false negative rates as being based on target lead levels at 
and above 1.4 mg/cm’ with confirmed values not to exceed 1.2 mg/cm’. Because confirmed 
lead levels of particular PEMs did not sometimes match target concentrations for those PEMs, 
false negative rates were assessed on panels with confirmed lead levels at 1.2 mg/cm” and 
higher. Consistent with the EPA’s April 22, 2008 RRP rule, panels with an ICP-AES confirmed 
lead level between 1.0 and 1.2 mg/cm’ were not used in the false negative analysis. 


Based on stakeholder technical panel input, the EPA lead paint action level of 1.0 mg/cm? lead 
was included for analysis as part of the verification test. Though evaluations of test kit 
performance based on this level is not in the EPA RRP rule, false positive and negative rates, in 
addition to those stated above, were also calculated for each test kit based on 1.0 mg/cm? lead. 
Thus, false positive rates were assessed on PEMs with confirmed lead levels at 1.0 mg/cm’ and 
lower and false negative rates were assessed on PEMs with confirmed lead levels at 1.0 mg/cm? 
and higher. For panels that measure 1.0 mg/cm’, positive results were considered “correct” and 
negative results were considered false negative. If the confirmed lead concentration of the PEM 
was greater than 1.0 mg/cm? (e.g., 1.1 mg/cm’), then negative results were considered false 


negatives. If the confirmed lead concentration of the PEM was less than 1.0 mg/cm’ (e.g., 0.9 
mg/cm”), then positive results were considered false positives. 


False positive and negative rates were calculated as shown in Equations | and 2, respectively: 


# of positive results 


False Positive Rate = @$2@@-—@£ ——__{_!—________________ 
total # of PEMs with lead level below 0.8 (or 1.0) mg/cm 


# of negative results (2) 
total # of PEMs with lead level above 1.2 (or 1.0)mg/cm’ 





False Negative Rate = 


5.2 Precision 


Precision was measured by the reproducibility of responses for replicate samples within a group 
of PEMs. Precision results were reported as the percentage of consistent responses from all 
replicate sets for those paint types (see Equation 3). Responses were considered inconsistent if 
25% or more of the replicates differed from the response of the other samples in the same group 
of PEMs.. 


#of consistent responses of replicatesets . 


Precision (% consistent results) = 100 (3) 


total number of replicate sets 


5.3 Sensitivity 


The sensitivity or lowest detectable lead level for each test kit was identified based on the 
detection results across all PEM lead levels. The lowest PEM lead level with consistent (>75%) 
positive or “detect” responses was considered the lowest detectable level. The identified lowest 
detectable lead level was reported and discussed. 


5.4 Modeled Probability of Test Kit Response 


Logistic regression models were used to determine the probabilities of positive or negative 
responses of the test kit at the 95% confidence level, as a function of lead concentration and 
other covariates, such as substrate type, lead paint type, operator type, and topcoat color. An 
evaluation of the bivariate relationship between the response variable and each candidate 
explanatory variable was performed by fitting single covariate logistic models to assess the 
predictive ability of each of the PEM parameters. Using the results from these bivariate 
analyses, a parsimonious multivariate model was developed including a set of explanatory 
variables which were most predictive of the probability of the test kit response variable. The 
potential logistic regression model took the form below: 


logit (Pr(Y, =1))=X,B (4) 


where Y; is the outcome of the test kit, X; is a vector of explanatory variables associated with Y; 
and f represent a vector of unknown parameters which was estimated with the model. Test 
results that indicated that lead was present were represented with Y=1; negative results were 
represented with Y=0. Candidate independent variables associated with the response variable 
were lead level (continuous), operator type (categorical), lead type (categorical), substrate type 
(categorical), and topcoat color (categorical). Interactions between categorical predictor 
variables were also assessed. Categorical covariates were modeled using indicator variables. 


SAS’s PROC LOGISTIC was used to evaluate the association between each explanatory variable and 
the probability of a positive test kit result. Then multivariable models were fit using a backward 
selection process whereby all explanatory variables were included in the initial model. In a multi- 
step backwards elimination process, the variable with the weakest association (highest Type III p- 
value) was eliminated from the model until all of the variables that remained had Type III p-values 
less than 0.05. The list of variables that remained formed the basis for evaluating interactions. 
Measured lead level was retained as an explanatory variable in all multivariable models. Two-way 
interactions were tested between all pairs of categorical explanatory variables that had p-values 
below 0.05. Interactions were retained in the multivariable models if their p-values were smaller 
than 0.05. 


5.4.2 Accounting for Measurement Error — SIMEX Background and Intuition 


Categorical covariates in this experiment were measured without error, but the lead level 
measurements were subject to some measurement error due both to variability inherent in the 
measurement (ICP-AES) process and possibly due to spatial heterogeneity in lead concentrations 
in paint on the PEMs themselves. The experimental design did not include multiple ICP-AES 
analyses per PEM so there is no direct estimate of the variability in measurements for these data. 
To account for the uncertainty associated with that error, the final multivariable model for each 
test kit was subjected to a simulation and extrapolation (SIMEX) analysis.'”'° 


A detailed description of SIMEX is beyond the scope of this report, but in short, it is a robust 
method of accounting for measurement error. The method requires either replicate measures of 
the quantity that is measured with error, or a characterization of the variability in the 
measurements. It then estimates what the regression model coefficients would be in the absence 
of measurement error. The technique estimates standard errors for the regression model 
coefficients using the bootstrap technique. SIMEX analyses were carried out in Stata version 
11.1 using the programs described in Hardin et al (2003c).'° 


The premise of the analysis is that one of the independent variables, namely lead concentration, 
has been measured with error. In the logistic regression models considered here, lead 
concentration is the only continuous independent variable; all of the other covariates are 
categorical. Thus, lead concentration may be considered the ‘x’ in a simple linear regression. 
The observed variability in ‘x’ is comprised of two components, actual variation in lead 


concentration and measurement error. If we were able to remove the measurement error then we 
would observe less variability in that independent variable. 


There are two important points of intuition that will inform expectations about what is seen in the 
SIMEX results. First, the data along the x-axis of a scatterplot would “tighten up” if 
measurement error were removed. “Tightening up” the independent variable in a regression 
analysis will result in a steeper slope or a regression coefficient with a larger magnitude. This is a 
fundamental consequence of any technique that adjusts for measurement error in the independent 
variable in a regression analysis. In the lead paint analysis, steeper logistic regression curves 
will result from the SIMEX analysis than would result from a non-SIMEX analysis where lead 
levels were considered to be fixed and known. 


Second, when the statistical analyses acknowledge and account for the measurement error, then 
the regression output prediction intervals may be wider than those for a non-SIMEX analysis 
where ‘x’ is considered to be fixed and known. For any given predicted value of the outcome 
variable, the prediction interval will most likely be wider, or at least not narrower. But for a 
fixed value of ‘x’, (such as 0.8 or 1.2 mg/cm’) whether the SIMEX prediction intervals are wider 
or narrower than the non-SIMEX intervals depends on how much the slopes of the SIMEX and 
non-SIMEX regression line differ. For typical logistic regression models, prediction intervals 
are very narrow at the extreme low and high asymptotic ends of the x-axis, and only appreciably 
wide in the region where the probability of the outcome is not near zero and not near one. So if 
the SIMEX analysis has only a moderate impact on the slope then wider prediction intervals 
might be observed at 0.8 and 1.2 mg/cm’. But if the slope changes dramatically, then 0.8 or 1.2 
mg/cm’ might now be in the part of the prediction curve that is near zero or one and the SIMEX 
prediction interval might be dramatically more narrow than a non-SIMEX< interval. 


Thus, the prediction curves for every SIMEX analysis are expected to be steeper than, or at least 
not less steep than, a non-SIMEX analysis. However, the assessment of test kit performance is 
based on the upper and lower bounds of prediction intervals at 0.8 and 1.2 .mg/cm‘’ respectively. 


5.4.3 SIMEX Input and Analysis 


During pre-production of the PEMs, replicate paint chip samples were analyzed from selected 
metal PEMs that served as reference panels (see Appendix A). Three metal panels were 
prepared for the pre-production homogeneity testing. Four paint chip samples, one from each 
quadrant of the PEM, were taken and analyzed via ICP-AES for their lead levels. Data are 
available on the coefficients of variation for these metal PEMs for both white and yellow lead. 
These data are shown below in Table 5-1. Though these data did not come from actual PEMs 
used during the lead paint test kit verification test, this information was used as a surrogate 
measure of homogeneity variability on the PEMs. 


For each PEM in the study, nine random pseudo-replicates were generated from a normal 
distribution with a mean equal to the confirmed lead concentration for that panel, and a standard 
deviation computed from the metal reference PEM data in Table 5-1 and indexed by the panel’s 
lead type and target lead level. The nine measurements were used as inputs to the Stata SIMEX 
algorithm as if they were true replicate measurements. 


Table 5-1. Results from Final Homogeneity Testing for each Set of ETV PEMs 


Target Lead Mean Levels CoV* 

Lead Type Level ICP (mg/cm?) ICP 
eee 
, F140 | 099 | 39 
ree 
reo | 597 | 142 
ros | 030 | 96 
ros | 062 | 44, 
P40 | 1.07 | 41.0) 
ee 
r 60 | 688 | 52 


* Coefficient of Variation (Standard Deviation/Mean x 100) 





There are two user-specified parameters for the Stata SIMEX algorithm: 1) the number of 
replicate measurements for the covariate measured with error, and 2) the number of bootstrap 
samples used to estimate standard errors on regression parameters. In testing not detailed here, 
the sensitivity of the SIMEX algorithm to different settings of these parameters was investigated. 
It was determined that the qualitative results were not sensitive to the values used in the analysis. 
The values used were nine pseudo-replicates per PEM and 199 bootstrap samples, respectively. 


The predicted regression curves and associated prediction intervals were generated in the interval 
0.0 to 6.0 mg/cm” using Stata. The relevant prediction bounds (the upper bound at 0.8 mg/cm” 
and lower bound at 1.2 mg/cm’) were assessed and the predicted false positive and false negative 
rates based on these prediction bounds were determined. 


5.4.4 Goodness of Fit 


To assess whether the logistic regression models fit the data well, standardized Pearson residuals 
were computed for every observation and those with an absolute value greater than two were 
flagged and plotted versus lead level. Standardized Pearson residuals greater than two are 
associated with observations that are not well fit by the model. In the logistic regression context 
observations that are not well fit might be those with high lead levels where the kit results were 
negative or very low lead levels where the kit results were positive. In the absence of categorical 
variables the standardized Pearson residuals should be normally distributed, so we would expect 
approximately 5% of the observations to have residuals with absolute value greater than two. In 
this case there are categorical covariates so the residuals are not strictly expected to be 
distributed normally but the proportion of observations with large residuals is still informative. 
That proportion is reported in Section 6.4. 
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5.5 Matrix Effects 


The covariate-adjusted logistic regression model described in Section 5.4 was used to assess the 
significance of PEM parameters and the interactions among them on the performance of the test 
kits. PEM parameters were included in the model as explanatory variables associated with the Y; 
response variable. 


Comparison of the observed values of the response variable to predicted values obtained from 
models with and without the predictor variable in question was the guiding principle in the 
logistic regression model. The likelihood function is defined as 


L(p)=[ 2, ).1- 2%) (5) 


where 7(Y,) is the conditional probability of Y;=1 and [1—7(Y,)] is the conditional probability 
of Y;=0 given the vector of explanatory variables (X). For purposes of assessing the 
significance of a group of p predictor variables (where p can be 1 or more), we computed the 
likelihood ratio test statistic, G, as follows: 


G = -2 log, [likelihood without the p variables / likelihood with the p variables] (6) 


Under the null hypothesis, this test statistic followed a chi-square distribution with p degrees of 
freedom. If the test statistic was greater than the 95th percentile of the chi-square distribution, 
then the group of variables, taken together, were statistically significant. 


5.6 Operational Factors 


There were no statistical calculations applicable to operational factors. Operational factors were 
determined qualitatively based on assessments from the Operator (both technical and non- 
technical) and the Battelle Verification Test Coordinator. Operational factors such as ease of 
use, operator bias, average cost, average time for kit operation, and helpfulness of manuals, were 
determined. Sustainability metrics such as volume and type of waste generated from the use of 
each test kit, toxicity of the chemicals used, and energy consumption are discussed. This 
discussion is based on how much waste was generated and what the waste was composed of, 
information from the vendor on how the waste should be properly handled, a summary of the 
pertinent MSDS information, when available, and noting whether the test kit used batteries, a 
power supply, or no energy source was needed. Information on how many tests each kit could 
perform as well as the shelf life of the test kit and chemicals used as part of the test kit was also 
reported. 
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Chapter 6 
Test Results 


The results for the LeadPaintCheck test kit are presented below for each of the performance 
parameters. The results from this test kit relied on the comparison of the absorbance reading to a 
conversion chart provided in the test kit instructions. Results were then categorized, based on 
the range the absorbance value fell into, as either, Negative (lead below 1.0 mg/cm’), Borderline, 
Positive for Lead (lead at 1.0 mg/cm’), Positive for Lead (lead at about 1.5 mg/cm”) and High 
Positive for Lead (at about 2.0 or 3.0 mg/cm’). Results that were Borderline were considered 
inconclusive and the sample was re-evaluated. The average absorbance value for the sample was 
then calculated and this number was used to determine the positive or negative result in the 
conversion table. If the result was again Borderline, the sample was considered negative, per the 
test kit instructions. 


In this report each PEM is associated with three definitions of lead levels: 

e Target lead level - the expected concentration of each PEM as outlined in Table 3-1. 
These target lead levels were 0, 0.3, 0.6, 1.0, 1.4, 2.0, or 6.0 mg/cm’. 

e Confirmed lead level - the concentration as measured by the reference laboratory using 
ICP-AES analysis. 

e Closest target lead level - the target level that is closest to the confirmed level. If a panel 
has a target lead level of 1.4 mg/cm? and a confirmed lead level of 1.9 mg/cm” then the 
closest target level is 2.0 mg/cm’. 





Under ideal circumstances the confirmed lead level would equal the target lead level, but this 
was sometimes not the case. Analyses where lead level was a categorical variable (.e., 
consistency, precision, and sensitivity analyses) characterized the panels by their closest target 
lead level. Analyses where lead level was a continuous variable (1.e., the false positive/negative 
and logistic regression analyses) characterized the panels by their confirmed lead level. Each 
analysis described clearly which level was used to characterize the lead level. 


6.1 False Positive and False Negative Rates 


Observed false positive and negative rates were calculated based on confirmed lead levels as 
measured though ICP-AES analysis. For example, if the PEM was confirmed to have a lead 
level of 1.4 mg/cm’, and the test kit returned a negative result, this would be considered a false 
negative. Table 3-1 details the target lead levels for the PEMs and the number of PEMs that 
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were anticipated at each lead level. Because of variations in PEM production, the confirmed 
lead level of a particular PEM did not always match the target lead level. Table 6-1 compares 
the number of PEMs at the confirmed and target lead levels used for the observed false positive 
and negative analyses. The data are divided into three categories: those panels eligible for false 
positive analysis (lead levels up to and including 0.8 mg/cm’), those excluded from false positive 
and false negative analyses (lead levels between 0.8 and 1.2 mg/cm’) and those eligible for false 
negative analysis (lead levels 1.2 mg/cm? and above). If the confirmed lead levels had been 
equal to the target lead levels, all of the numbers would lie along the shaded diagonal. Because 
the confirmed levels sometimes differed significantly from the target levels, (i.e., the target lead 
level was at 0.6 mg/cm” but confirmed near 1.4 mg/cm”) some panels appear in the off-diagonal 
table entries and were therefore included in portions of the analysis other than those for which 
they had been targeted. 


Table 6-1. The number of panels in each false positive and false negative analysis category 


Confirmed Lead Levels 

















Eligible Eligible 

for False for False 

Positive Excluded from Negative 
Analysis Analysis Analysis Total 
Target | Eligible for False Positive Analysis 147 21 12 180 

Lead Excluded from Analysis 7 40 24 71 

Levels | Eligible for False Negative Analysis 0 13 203 216 
Total 154 70 239 467 


Tables 6-2 and 6-3 list the observed false positive and false negative rates for the 
LeadPaintCheck test kit under two sets of conditions: 
e Table 6-2 shows the observed false positive results for panels with confirmed lead levels 
< 0.8 mg/cm? and observed false negative results for panels with confirmed lead levels = 
1.2 mg/cm”, per the RRP ruling’. 
e Table 6-3 shows observed false positive results for panels with confirmed lead levels < 1 
mg/m’ and observed false negative results for panels with confirmed lead levels > 1 
mg/cm’. 


Results for both the technical and non-technical operator are presented. Results are presented as 
overall rates (i.e., false positive and negative results across all applicable PEMs combined) and 
also false positive and negative rates based on lead paint type (1.e., white or yellow lead), 
substrate (i.e., drywall, metal, plaster, or wood), and topcoat paint color (i.e., grey red or white). 


The overall observed false negative rate for the LeadPaintCheck test kit for the technical operator 
based on confirmed lead levels of >1.2 mg/cm” was 2% while the overall observed false negative 
rate for the non-technical operator was 1% (see Table 6-2). As Table 6-2 shows, observed false 
negative rates determined by substrate, lead type, and topcoat color were similar to the overall 
false negative rates and ranged from 0 — 3% across operator. 
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Observed false positive rates, for this same RRP rule grouping of the PEMs, however, were 
higher. The overall observed false positive rate for the technical operator was 14% while that for 
the non-technical operator was 16% (see Table 6-2). For both the technical and non-technical 
operator, observed false positive rates ranged from 7% to 24% and were comparable to the 
overall rate across all other evaluated subgroups (lead paint type, substrate, and topcoat color). 


Table 6-2. LeadPaintCheck test kit false positive results for panels with confirmed lead 

levels < 0.8 mg/cm’ and false negative results for panels with confirmed lead levels > 1.2 
2 

mg/cm 














LeadPaintCheck Test Kit 
False Positives’ False Negatives" 

Technical Non-technical Technical Non-technical 

Operator Operator Operator Operator 
Overall 44 / 308 = 14% 49 / 308 = 16% 8/478 =2% 4/478 =1% 
None 0/72=0% 0/72=0% NA NA 
White 24/128 =19% 30 / 128 = 23% 0/234 =0% 3/234=1% 
Yellow 20 / 108 = 19% 19 / 108 = 18% 8 / 244 = 3% 1/244 =0% 
Drywall 12 /72=17% 17/72 = 24% 0/124 =0% 1/124=1% 
Metal 10 / 98 = 10% 10 / 98 = 10% 1/94=1% 1/94=1% 
Plaster 4/54=7% 5/54=9% 5/146 = 3% 0/146 =0% 
Wood 18 /84=21% 17 / 84 = 20% 2/114=2% 2/114=2% 
Grey 14/106 =13% 18/106 =17% 1/148 =1% 0/148 =0% 
Red 16 / 102 = 16% 18 / 102 = 18% 3/168 =2% 2/168 =1% 
White 14/100 = 14% 13 / 100 = 13% 4/162 =2% 2/162=1% 








‘False positives on PEMs with confirmed lead levels < 0.8 mg/cm? 


"False negatives on PEMs with confirmed lead levels > 1.2 mg/cm? 
NA: If the paint did not contain lead then a false negative is not possible, those entries are ‘NA’ 
(not applicable). 


The observed false negative rates for both the technical and non-technical operator using 1.0 
mg/cm? as the deciding concentration (see Table 6-3) were similar to those found using RRP rule 
concentration limits (see Table 6-2). The overall observed false negative rate for the technical 
operator was 3% while that for the non-technical operator was 2%. Observed false negative rates 
for the PEM categories were slightly higher than those in Table 6-2. The overall observed false 
positive rate for the LeadPaintCheck test kit, when calculated by dividing the PEMs at the 1.0 
mg/cm” level, was approximately twice as high for both operators as rates found using 0.8 
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mg/cm’ as the determining concentration. Observed false positive rate patterns similar to those 


seen in Table 6-2 were apparent across different PEM characteristics of interest. 


Table 6-3. LeadPaintCheck test kit false positive results for panels with confirmed lead 
levels < 1 mg/cm” and false negative results for panels with confirmed lead levels > 1 














mg/cm? 
LeadPaintCheck Test Kit 
False Positives' False Negatives" 

Technical Non-technical Technical Non-technical 

Operator Operator Operator Operator 
Overall 105/388=27% 113/388=29% 17 / 546 = 3% 10 / 546 = 2% 
None 0/72=0% 0/72=0% na na 
White 58 / 166 = 35% 62 / 166 = 37% 3/264=1% 4/264=2% 
Yellow 47/150 = 31% 51/150 = 34% 14/282 =5% 6 / 282 = 2% 
Drywall 30/94 =32% 39 /94= 41% 0/140 =0% 1/140=1% 
Metal 30/126 = 24% 29 / 126 = 23% 1/108=1% 1/108 =1% 
Plaster 12 /66=18% 12 /66=18% 12/168 =7% 3/168 =2% 
Wood 33 /102 = 32% 33 / 102 = 32% 4/130=3% 5/130=4% 
Grey 34 / 130 = 26% 38 / 130 = 29% 6 / 182 = 3% 3/182=2% 
Red 34/128=27% 38 / 128 = 30% 5/184 = 3% 3/184=2% 
White 37 / 130 = 28% 37 / 130 = 28% 6 / 180 = 3% 4/180 =2% 





‘False positives on PEMs with confirmed lead levels < 1.0 mg/cm? 





"False negatives on PEMs with confirmed lead levels > 1.0 mg/ cm* 
NA: If the paint did not contain lead then a false negative is not possible, those entries are ‘NA’ 
(not applicable). 


Note that the observed false positive and negative rates presented in this section provide a 
general representation of the ability of the LeadPaintCheck test kit to correctly identify regulated 
lead paint when it is present or absent. The results presented in Table 6-2 provide rates based on 
the cut-off concentration (0.8 or 1.2 mg/cm”) as well as all levels evaluated below or above those 
concentrations. To evaluate test kit performance based on the RRP rule, lead paint test kits 
should have a demonstrated probability (with 95% confidence) of a negative response at or 
above the regulated lead level <5% of the time. Test kits should also have a demonstrated 
probability (with 95% confidence) of a positive response below the regulated lead level <10% of 
the time. Because the RRP rule also indicated that test kit performance would not be based on 
lead levels between 0.8 and 1.2 mg/cm’, the false positive and negative probabilities assessed in 


25 


this report were then based around the excluded concentrations (of 0.8 and 1.2 mg/cm’). False 
positive and negative rates associated with these criteria are discussed in Section 6.4. 


6.2 Precision 


To compute precision, it is first necessary to compute the number of replicate sets with consistent 
responses. Replicate sets are defined in the test/QA plan’ to be groups of panels with similar 
lead levels. The target lead levels in this experiment were 0, 0.3, 0.6, 1.0, 1.4, 2, and 6 mg/cm? 
but the lead levels that were achieved, as confirmed by ICP-AES, sometimes varied from those 
target levels. To assemble replicate sets that represented the target lead levels, the panels were 
assigned to the replicate set that was nearest their confirmed lead level. In other words, if a 
particular panel was targeted for 0.3 mg/cm” but was measured to have 0.9 mg/cm? then it was 
assigned to the replicate set nearest 0.9 mg/cm”, which was the set labeled 1.0 mg/cm”. Table 6- 
4 shows the thresholds that defined the replicate set bins as well as the range of measured levels 
that fell in each bin. 


Table 6-4. Actual lead levels and their replicate set labels 





Replicate Set Bin 





Label (mg/cm’) Bin Thresholds Confirmed Lead Levels In 
(Closest Target (mg/cm?) This Bin (mg/cm’) 
Lead Level) 
0 Targeted to have zero lead 0.000 - 0.065 
0.3 0 < Confirmed Lead Level < 0.45 0.141 - 0.449 
0.6 0.45 < Confirmed Lead Level < 0.8 0.454 - 0.7998 
1 0.8 < Confirmed Lead Level < 1.2 0.803 - 1.190 
1.4 1.2 < Confirmed Lead Level < 1.7 1.209 - 1.694 
2 1.7 <Confirmed Lead Level < 4 1.702 - 3.839 
6 4 <Confirmed Lead Level 4.757 - 15.33 











Table 6-5 shows the number of panels in which confirmed lead levels fell nearest their target 
level and the number of panels whose confirmed levels fell closer to a level other than their 
target level. The shaded values along the diagonal of the table are the panels in which measured 
levels fell closer to their target than to any of the other targets. If all of the panels had measured 
levels that were equal to their target levels, then all of the numbers would lie along the diagonal 
of Table 6-5. The numbers off the diagonal represent panels with confirmed lead levels closer to 
some other target value. Note, for example, that of the 71 panels that were targeted to have 1.0 
mg/cm? of lead, 40 achieved that level, seven fell closer to 0.6 mg/cm? than 1.0 mg/cm’, 19 fell 
closer to 1.4 mg/cm”, and five fell closer to 2.0 mg/cm” than to any other target level. In the 
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consistency analysis described below, each panel was grouped into sets labeled with the target 


level that its measured level fell closest to, rather than by its target lead level. 


Table 6-5. The number of panels at each target level and the number in each replicate set 


bin 


Replicate Set Bin 


(Target level that is closest to the panel’s actual measured lead level) 


0.3 
0.6 
1.0 
1.4 
2.0 
6.0 
Total 


Target Lead Level 
(mg/cm’) 


Table 6-6 lists consistency results for the LeadPaintCheck test kit by operator type, lead type, 
substrate, and lead level. Each table entry lists the number of test results with those 














0 0.3 0.6 1 1.4 2 6 Total 
36—Ci«C : : : So 36 
- 6 #10 1 ‘ - 2 72 
: 2 HW 20 8 4 .- 72 
: : 7 bw 5 - 71 
: : - 3 68 0 - 72 
: : : : 6 6 1 72 
- - - - - - 72 72 
36 63 55 74 82 84 73 467 


characteristics (N) as well as the proportion of the results that were positive for lead (Pos). Table 


entries where the proportion is below 25% or above 75% are ‘consistent’, meaning that more 
than three-quarters of the results were the same (negative or positive). Table entries where the 
proportion of positive results ranges from 25% to 75% are considered to be ‘inconsistent’. 


Inconsistent entries are shaded in the tables. Overall consistency results across all substrates for 
white and yellow lead panels for each operator type are also provided in the last row of Table 6- 
6. Results across both operators and lead paint types are provided in the last column of the table. 


Table 6-6 shows that the consistency of the LeadPaintCheck test kit at particular lead levels was 


similar across substrates and operator type. In fact, overall, across both operator types and lead 


paint types, as shown in the far-right column of Table 6-6, the LeadPaintCheck test kit provided 


consistent results for all lead levels except for 0.6 mg/cm”. Inconsistent results were also 


obtained for 1.0 mg/cm? lead on plaster PEMs, with positive results 74% of the time. Results for 


all other lead levels were highly consistent, between 93-100% or 0-8%. Only the evaluation of 
metal PEMs returned consistent results for all lead levels overall. Results for the non-technical 


operator showed fewer inconsistencies than those for the technical operator. 
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Table 6-6. LeadPaintCheck test kit consistency results by operator type, lead type, 


substrate, and lead level 


LeadPaintCheck Test Kit 



























































NON-TECHNICAL TECHNICAL TOTAL 
Lead Type None White Yellow Total None White Yellow Total Total 
N Pos N Pos N Pos N Pos N Pos N Pos | N Pos N Pos N Pos 
DRYWALL 
0 18 0% 18 0% | 18 0% 18 0% 36 0% 
0.3 20 10% | 16 13% 36 11% 20 10% | 16 0% 36 5% 72 8% 
0.6 8 88% | 10 60% 18 74% 8 50% | 10 60% 18 55% 36 64% 
1 20 100% | 18 100% 38 100% 20 95% | 18 83% 33 89% 76 95% 
1.4 18 100% | 28 100% 46 100% 18 100% | 28 100% 46 100% 92 100% 
2 24 96% | 18 100% 42 98% 24 100% | 18 100% 42 100% 84 99% 
6 18 100% | 18 100% 36 100% 18 100% | 18 100% 36 100% 72 100% 
METAL | ee eee | | ot 
0 18 0% 18 0% | 18 0% 18 0% 36 0% 
0.3 20 10% | 18 6% 38 8% 20 0% | 18 0% 38 0% 76 4% 
0.6 24 21% | 18 11% 42 16% 24 21% | 18 28% 42 24% 84 20% 
1 16 81% | 26 77% 42 79% 16 88% | 26 77% 42 82% 84 81% 
1.4 14 100% | 14 100% 28 100% 14 100% | 14 93% 28 96% 56 98% 
2 16 94% | 14 100% 30 97% 16 100% | 14 100% 30 100% 60 98% 
6 18 100% | 18 100% 36 100% 18 100% | 18 100% 36 100% 72 100% 
PLASTER | | 
0 18 0% 18 0% | 18 0% 18 0% 36 0% 
0.3 18 6% 4 0% 22 3% 18 11% 4 0% 22 6% 44 4% 
0.6 2 100% | 12 17% 14 58% 2 50% | 12 8% 14 29% 28 44% 
1 12 83% | 22 73% 34 78% 12 75% | 22 64% 34 69% 68 74% 
1.4 22 100% | 26 100% 48 100% 22 100% | 26 81% 48 90% 96 95% 
2 36 100% | 24 100% 60 100% 36 100% | 24 100% 60 100% | 120 100% 
6 18 100% | 20 100% 38 100% 18 100% | 20 100% 38 100% 76 100% 
0 18 0% 2 0% 20 0% | 18 0% zZ 0% 20 0% 40 0% 
0.3 16 0% | 12 0% 28 0% 16 0% | 12 0% 28 0% 56 0% 
0.6 18 61% | 18 33% 36 47% 18 56% | 18 44% 36 50% 72 49% 
1 20 90% | 14 79% 34 84% 20 95% | 14 71% 34 83% 68 84% 
1.4 12 100% | 30 100% 42 100% 12 100% | 30 93% 42 97% 84 98% 
2 20 100% | 16 94% 36 97% 20 100% | 16 100% 36 100% 72 98% 
6 18 94% | 18 100% 36 97% 18 100% | 18 100% 36 100% 72 99% 
ALL | 
0 | 72 0% | | 74 0% | 72 0% 74 0% | 148 0% 
0.3 74 7% | 50 6% | 124 6% 74 5% | 50 0% | 124 2% | 248 4% 
0.6 52 48% | 58 28% | 110 41% 52 38% | 58 34% | 110 38% | 220 40% 
1 68 90% | 80 81% | 148 85% 68 90% | 80 74% | 148 81% | 296 83% 
1.4 66 100% | 98 100% | 164 100% 66 100% | 98 92% | 164 96% | 328 98% 
2 96 98% | 72 99% | 168 98% 96 100% | 72 100% | 168 100% | 336 99% 
6 72 99% | 74 100% | 146 99% 72 100% | 74 100% | 146 100% | 292 100% 





N = number of test results in each bin of the table 


Pos = Proportion of those N test results that were ‘Positive’ for the presence of lead. 
Lead levels in the left-most column represent the target level closest to the measured level of lead in the panel. 
Shaded cells represent ‘inconsistent’ results. i.e.,% positive is between 25% and 75% 
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The consistency results provided in Table 6-6 were used to calculate precision. Precision was 
estimated for panels with no lead, white lead, and yellow lead and broken out by type of operator 
and then aggregated across both types of operators. For any column in Table 6-6, the precision 
is simply the proportion of consistent (unshaded) table entries in the rows for the four different 
substrates. The ‘All’ rows are not counted in the precision calculation because those table entries 
are summaries of the entries for the four substrates. Thus, precision was calculated as: 


bef aurteded tabta sai ter in tea 


deywellmatel plaster and weed ssettone 
Precision(%6 consistent results) = S2@sliestel plaster and weed sections (7) 


Table 6-7 lists the results of the precision calculations for the LeadPaintCheck test kit. Higher 
proportions of consistent results indicate more consistency and higher precision. 


Table 6-7. LeadPaintCheck test kit precision results by lead type and operator type 


No Lead White Lead Yellow Lead 








Non-technical 4/4=100% 24/25=96% 21/24 = 88% 
Technical 4/4=100% 21/25=84% 19/24=79% 
All 8/8 =100% 45/50=90% 40/48 = 83% 





The LeadPaintCheck test kit was precise (100%) in evaluating PEMs with no lead. Because of 
the strong consistent results noted for the LeadPaintCheck test kit (see Table 6-6), the precision 
of this test kit was high across both lead paint types (see Table 6-7). For white lead PEMs, the 
LeadPaintCheck test kit produced consistent responses 90% of the time. For yellow lead PEMs, 
the LeadPaintCheck test kit was precise 83% of the time. Operator type was related to the 
precision of the results for both the yellow and white lead PEMs. The precision of the non- 
technical operator was greater than that of the technical operator for both paint types (96% vs. 
84% for white lead and 88% vs. 79% for yellow lead). 


6.3 Sensitivity 


Sensitivity was calculated using the bottom six rows in Table 6-6. These rows aggregate results 
across all four substrates. For the white lead and yellow lead columns in these tables, the 
sensitivity is the lowest lead level = 1 mg/cm? that is consistently detected with positive results 
(unshaded and > 75%). Ideally the kit would give consistently negative results for lead levels < 
1 mg/cm’ and consistently positive results for levels > 1 mg/cm’ so the optimal sensitivity results 
would be | across every row of Table 6-8. Note that the LeadPaintCheck test kit is qualitative in 
nature and the goal of the test kit is to indicate to the end user if lead paint is present or not at the 
level of 1.0 mg/cm’. 
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Table 6-8. LeadPaintCheck test kit sensitivity results — lowest lead level for which the kit 
gave consistent positive results (mg/cm”) 

















Non-technical Operator Technical Operator All 
Lead Type | White Yellow Total White Yellow Total Total 
Sensitivity 1 1 1 1 1.4 1 1 











The overall sensitivity of the LeadPaintCheck test kit was at the 1.0 mg/cm’ lead level. The kit 


provided consistent positive responses at 1.0 mg/cm’ across both operators and lead paint types 
in all cases except for yellow lead PEMs evaluated by the technical operator. In this case, the 


test kit was only sensitive down to 1.4 mg/cm’. 


6.4 Modeled Probability of Test Kit Response 


Table 6-9 lists the explanatory variables which had significant (p<0.05) univariate associations 
with the probability of obtaining a positive test kit result. Only lead level and substrate type 


explanatory variables showed a statistically significant univariate association with the probability 


of a positive response. Only these variables were retained in the multivariable model after 
backward selection. Table 6-10 lists the parameter estimates for the multivariable logistic 


regression models from the Stata SIMEX program. 


Table 6-9. LeadPaintCheck test kit univariate associations between probability of positive 


response and explanatory variables 





Explanatory Variable Significant Univariate Association? 


Included in Multivariable Model? 





Substrate Type 
Topcoat Color 





Lead Level Yes (p-value < 0.0001) 
Lead Type No (p-value = 0.9785) 
Operator Type No (p-value = 0.4146) 


Yes (p-value < 0.0001) 
No (p-value = 0.9153) 


Yes 
No 
No 

Yes 
No 
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Table 6-10. LeadPaintCheck test kit multivariable Stata SIMEX logistic regression 
parameter estimates 


Simulation extrapolation No. of obs 
Bootstraps reps 


Residual df = Wald F(4,1863) 
Prob > F 


Variance Function: u (l=) [Bernoulli] 
Link Function : log (u/ (1-u) ) 





Bootstrap 
result ] E [95% Conf. Interval 











Substrate : (wood is the referenc 
drywall -4088628 -2538184 t ; —.0889355 - 9066611 
metal —.54044 -2386198 ‘ 0. -1.00843 —.0724497 


plaster —.957398 -3791874 : j -1.701075 =213-7212 





lead level 5.870011 - 7731209 a : 4.353737 7.386286 
constant -4.045913 ~92829 : . -5.082016 -3.009811 














Table 6-11 lists the modeled probability of a positive test result for the LeadPaintCheck test kit 
when the lead level is 0.8 mg/em? (PREDICTION) along with the upper bound of a 95% 
prediction interval (UPPER). That upper bound can be considered to be a worst-case estimate of 
the false positive probability when the true lead level is 0.8 mg/cm? (FALSE POS RATE). 
Ideally, for purposes of the RRP rule, the numbers in the UPPER/FALSE POS RATE column 
would be < 10%. Note that the FALSE POS RATE in Table 6-11 is higher than those in Tables 
6-2 and 6-3. In those earlier tables the rates considered panels at a variety of comparatively low 
lead levels so some cases should have been easier for the kit to obtain the correct answer. In 
Table 6-11, the false positive rate is evaluated only at 0.8 mg/cm’ so the rate does not benefit 
from the comparatively lower lead concentrations. Evaluating at only this level also ensures that 
a test kit can adequately perform at concentrations of lead paint closest to the current regulatory 
level. 


The modeled probability curve results, as shown in Table 6-11, indicate that at 0.8 mg/cm’, there 
is no substrate type where the upper prediction bound provides a false positive rate of <10%. In 
fact, the lowest upper prediction bound at 0.8 mg/cm? was 53% for plaster. 


Table 6-12 lists the modeled probability of a positive test result for the LeadPaintCheck test kit 
when the lead level is 1.2 mg/em? (PREDICTION) along with the lower bound of a 95% 
prediction interval (LOWER). The difference between the lower bound and 100% can be 
considered to be a worst-case estimate of the false negative probability when the true lead level 
is 1.2 mg/cm? (FALSE NEG RATE). Ideally the numbers in the FALSE NEG RATE column 
would be < 5%. 


Note that the FALSE NEG RATE in Table 6-12 is higher than those in Tables 6-2 and 6-3. In 


the earlier tables, the false negative rates considered panels at a variety of comparatively high 
lead levels so some cases should have been easier for the kit to obtain the correct answer. In 
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Table 6-12, the false negative rate is evaluated only at 1.2 mg/cm‘ so the rate does not benefit 
from the comparatively higher lead concentrations. Evaluating at only this level also ensures 
that a test kit can adequately perform at concentrations of lead paint closest to the current 
regulatory level. 


None of the false negative rates in Table 6-12 was below 5%. The lowest rate is 7% for drywall 
and the highest is 18% for plaster. 


Table 6-11. LeadPaintCheck test kit modeled probability of positive test results and upper 
95% prediction bound when lead level = 0.8 mg/cm? 











UPPER 
SUBSTRATE LEAD LEVEL PREDICTION (FALSE POS RATE) 
DRYWALL 0.8 74.2% 80.9% 
METAL 0.8 52.7% 59.2% 
PLASTER 0.8 42.4% 53.3% 
WOOD 0.8 65.7% 73.3% 








Table 6-12. LeadPaintCheck test kit modeled probability of positive test results, lower 
95% prediction bound, and corresponding conservative estimate of the false negative rate 
when lead level = 1.2 mg/cm? 











SUBSTRATE LEAD LEVEL PREDICTION LOWER FALSE NEG RATE 
DRYWALL 1.2 96.8% 93.1% 6.9% 
METAL 1.2 92.1% 86.2% 13.8% 
PLASTER Ae 88.5% 81.8% 18.2% 
WOOD 12 95.2% 90.3% 9.7% 








As another means of reporting the results for the LeadPaintCheck test kit, modeled probability 
curves were also plotted based on the results of the regression analysis. To better understand the 
information being provided in these probability curves, a brief explanation is presented here. 
Figure 6-1 shows that for the perfect or ideal test kit, the probability of a positive test result 
would be a step function. The probability of a positive result would be zero below 1.0 mg/cm? 
and 100% at or above 1.0 mg/cm’. Under the RRP rule, a test kit must yield a demonstrated 
probability (with 95% confidence) of no more than 10% false positives at lead concentrations 
below 0.8 mg/cm? and a demonstrated probability (with 95% confidence) of no more than 5% 
false negatives at concentrations above 1.2 mg/cm’. Figure 6-1 also shows a performance curve 
for a hypothetical test kit that achieves those rates. The upper bound of the 90% prediction 
interval is at 10% at 0.8 mg/cm” and the lower bound of the prediction interval is at 95% at 1.2 
mg/cm’. 
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One way to think of the test kit performance guidelines is in terms of regions of the probability 
plots. Figure 6-2 demonstrates this concept. For the kit to be within limits set up by the RRP 
rule, the probability curve must trace a path through the white region in the figure and must not 
stray into the shaded regions. If the curve crosses the shaded region at the left side of the graph 
then there are lead levels < 0.8 mg/cm’ where the false positive rate is > 10%. If the curve 
crosses the shaded region at the right side of the graph then there are lead levels > 1.2 mg/cm? 
where the false negative rate is > 5%. Either type of intersection between the curve and the 
shaded region indicates that the kit does not meet the performance levels stipulated in the RRP 
rule. 


Note that results for the region between 0.8 and 1.2 mg/cm’ were not discussed in this report. 
This is consistent with the RRP rule stipulation that lead concentrations between 0.8 and 1.2 
mg/cm’ were not to be considered for the evaluation of the performance of lead paint test kits. 


Figures 6-3 shows the predicted probability of obtaining a positive test result using the 
LeadPaintCheck test kit over the full range of explanatory variables along with the bounds of a 
90% prediction interval. Note that the upper and lower bounds of the 90% prediction interval 
may also be considered to be upper and lower 95% prediction bounds for one-sided inference. 


Note that in all four panels of Figure 6-3 the black curves intersect the shaded regions below 0.8 
mg/cm’. This indicates, with 95% confidence, that there are lead levels below 0.8 mg/cm? where 
the false positive rates would be higher than 10%, which do not agree with the criteria of the 
RRP rule. Note also that in all four panels the black prediction interval curves intersect the 
shaded region above 1.2 mg/cm’. This indicates that there are lead levels above 1.2 mg/cm? 
where the predicted false negative rate is greater than 5%, which also do not agree with the 
criteria of the RRP rule. 
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Figure 6-1. Probability curves that represent test kit results that are both perfect (red line) and within RRP rule criteria (black 
solid line). 
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Figure 6-2. Probability curves with shaded region to denote performance results that meet RRP rule false positive and 
negative criteria. Test kits with curves that fall within the white region and avoid the shaded region meet the RRP rule. 
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Figure 6-3. LeadPaintCheck test kit predicted probability of positive test result (solid lines) with 90% prediction interval 
(dotted lines) for various substrates. 
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Based on the modeled probabilities shown in Figure 6-3, threshold values for false positive and 
negative rates were established for the LeadPaintCheck test kit. For the false positive rate, this 
threshold value is the lead level, with 95% confidence, below which the LeadPaintCheck test kit 
would yield fewer than 10% false positive results. For the false negative rate, this threshold 
value is the lead level, with 95% confidence, above which the LeadPaintCheck test kit would 
yield fewer than 5% false negative results. These threshold values are then the lead levels where 
the LeadPaintCheck test kit is predicted to meet the false positive and negative criteria set forth 
in the RRP rule. 


Table 6-13 presents the false positive and negative threshold values for the LeadPaintCheck test 
kit. Threshold lead levels are provided for each substrate. 


Table 6-13 indicates that overall, across all factors, the false positive threshold is 0.24 mg/cm? 
and the overall false negative threshold is 1.40 mg/cm” for the LeadPaintCheck test kit. The 
threshold values for the individual substrates are similar to the overall values. 


Table 6-13. LeadPaintCheck test kit false positive and negative threshold values (95 % 
confidence) based on the modeled probability of test results 

















FALSE POSITIVE FALSE NEGATIVE 
SUBSTRATE THRESHOLD (mg/cm?) | THRESHOLD (mg/cm?) 
DRYWALL 0.12 1.28 
METAL 0.30 1.44 
PLASTER 0.33 1.50 
WOOD 0.20 1.36 
OVERALL 0.24 1.40 








Standardized Pearson residuals were calculated to assess goodness of fit of the logistic regression 
models. For the LeadPaintCheck test kit model, 95.3% of the residuals had absolute values 
smaller than two. 


6.5 Matrix Effect 


The matrix effects for the LeadPaintCheck test kit were evaluated with results in Table 6-10. 

The variables that were retained in the multivariable logistic regression model each add 
significant explanatory power to their respective models. Those variables are significantly 
associated with the probability of obtaining a positive test result from the kits tested in this study. 


For the LeadPaintCheck test kit, Table 6-10 indicates that after controlling for the significant 


covariates, the likelihood of a positive test result is positively and significantly associated with: 
higher lead levels and metal and plaster substrates. 


a 


6.6 Operational Factors 


Both the technical and non-technical operator found the LeadPaintCheck test kit instructions to 
be clear, informative, and easy to follow. Step-by-step instructions were provided in the manual 
with color pictures of each step being completed. Details on the operation and maintenance of 
the eXact® LEADQuick™ Photometer were included in the instruction manual. An instructional 
DVD was provided with the instructions. The non-technical operator received no training from 
the vendor and relied solely on the test kit instructions and the informational DVD for his 
understanding of the operation of the test kit. Both the technical and non-technical operator 
stated that the instruction manual and DVD provided good guidance for readily being able to 
operate the test kit, although the non-technical operator’s opinion was that the kit was 
inconvenient for field use due to the steps involved. 


The solutions used for different steps were easily identifiable within the kit and the storage 
conditions of the reagents were readily marked. All reagents came prepared and ready to use. 


The LeadPaintCheck test kit as supplied for this verification test came with an eXact® 
LEADQuick™ Photometer, all necessary reagents (including the quality check standard and the 
photometer strips), a homogenizer kit by Omni that included all components necessary to operate 
the homogenizer, a 500 mL wash/squirt bottle, 15 mL plastic conical tubes with lids, 50 mL 
plastic conical tubes with lids, ¥2 inch cork bore, cork bore sharpener, 100 uL pipette and 
associated disposable tips, % inch cork bore (for applications not part of this verification test), 
scissors, forceps, rod and base to hold the homogenizer, small brush, funnel system for paint chip 
collection, scalpel with blades, masking tape, and a cork bore remover. DI water and Static 
Guard® were not supplied with the test kit. The homogenizer came in its own case. All of these 
components approximate the LeadPaintCheck test kit startup kit, which supplies everything 
listed here in sufficient quantities for 50 tests. 


According to the vendor, the LeadPaintCheck test kit should be stored out of direct sunlight and 
away from chemical storage areas. Exposure to temperatures above 80° Celsius should also be 
avoided for the test reagents and the photometer. Per guidance from the vendor, the Omni tips 
for the homogenizer should not be used for more than 20 samples. As indicated in the test kit 
instructions, the diluted HP sample must be tested within 24 hours of preparation. The HP 
sample itself is recommended to be used within 30 days of preparation. Expiration dates were 
provided on the solution bottles. According to the labels, the solutions had a shelf-life of 1 year. 


The test kit instructions indicated that appropriate safety precautions should be taken when 
working with the test kit and lead paint and recommend wearing appropriate personal protective 
equipment, including safety glasses and gloves, which are not provided with the test kit. 

The eXact® Reagent Acid-1 contains 25% nitric acid, which can cause burns and react with 
metals to form hydrogen gas. The eXact® Reagent Pb-2 has an alkali pH of around 10.8 and can 
cause burns to the skin and eyes. The eXact® Strip Pb-3 can cause skin and eye irritation. 
Information on the hazards of these solutions, the chemicals of concern in each solution, the 
steps that should be taken to avoid injury, and first aid measures for each solution were provided 
in material safety data sheets (MSDS) supplied as part of the LeadPaintCheck test kit 
instructions. Both the technical and non-technical operators followed general laboratory safety 
procedures and wore a lab coat, protective eyewear, and gloves at all times. 
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The HP sample solution, diluted HP sample solution, rinses for the photometer, eXact® Strip Pb- 
3 strips, and pipette tips were produced as waste for a single test. The test kit instructions 
indicated that the 15 and 50 mL conical tubes could be re-used. For the purposes of this test, 
these tubes were thrown away after each use for convenience and to avoid any potential cross- 
contamination issues. If a positive result is obtained with the test kit, most of these components 
would be considered to contain lead. The eXact® Strip Pb-3 and pipette tips can be thrown away 
in the regular trash. According to the test kit instructions, the HP solutions (regular and diluted) 
should be flushed down the drain with copious amounts of water and the conical tubes should 
then be rinsed and allowed to dry before re-use. (Note: Because regulations for the disposal of 
wastes generated from the use of lead test kits may vary from state to state, EPA recommends 
that test kit users contact their state government agency for proper waste disposal requirements.) 
Rinses used for the photometer should be discarded into a sink and flushed with water. 


The LeadPaintCheck test kit provided clear and user-friendly instructions, but required multiple 
steps. Operation of the test kit took approximately 22 minutes for the technical operator and 
approximately 27 minutes for the non-technical operator, not including clean-up time. The non- 
technical operator considered the LeadPaintCheck test kit to be inconvenient for use. A regular 
power supply was needed for the operation of the homogenizer. Four AAA batteries were 
needed for the operation of the eXact® LEADQuick™ Photometer. The LeadPaintCheck Startup 
Kit is priced at $1999.99. The reagent replacement set is $99.99 for 50 tests. After the initial 
Startup Kit investment the cost per paint test is $2. 
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Chapter 7 
Performance Summary 


The overall observed false negative rate for the LeadPaintCheck test kit on PEMs with confirmed 
lead levels >1.2 mg/cm? was 2% for the technical and 1% for the non-technical operator. The 
overall observed false positive rate on PEMs with confirmed lead levels of < 0.8 mg/cm’ was 
14% for the technical operator and 16% for the non-technical operator. 


The LeadPaintCheck test kit provided overall consistent responses (either positive or negative) 
for both the technical and non-technical operator for all lead levels except 0.6 mg/cm’. At this 
level, responses were consistently positive 40-64% of the time. Inconsistent results were also 
obtained for 1.0 mg/cm? lead on plaster PEMs, with positive results 74% of the time. Results for 
all other lead levels were highly consistent, between 93-100% or 0-8%. Only the evaluation of 
metal PEMs returned consistent results for all lead levels overall. Results for the non-technical 
operator showed fewer inconsistencies than those for the technical operator. 


Results from the LeadPaintCheck test kit indicated 100% precision in evaluating PEMs with no 
lead. Because of the strong consistent results noted for the LeadPaintCheck test kit, the precision 
of this test kit was high across both lead paint types. For white lead PEMs, the LeadPaintCheck 
test kit produced consistent responses 90% of the time. For yellow lead PEMs, the 
LeadPaintCheck test kit was precise 83% of the time. 


The overall sensitivity of the LeadPaintCheck test kit was at the 1.0 mg/cm’ lead level. The kit 
provided consistent positive responses at 1.0 mg/cm? across both operators and lead paint types 
in all cases except for yellow lead PEMs evaluated by the technical operator. In this case, the 
lowest level for which the test kit provided consistent positive responses was 1.4 mg/cm’. 


Under the RRP rule’, a test kit must yield a demonstrated probability (with 95% confidence) of 
no more than 10% false positives at lead concentrations below 0.8 mg/cm” and a demonstrated 
probability (with 95% confidence) of no more than 5% false negatives at concentrations above 
1.2 mg/cm’ to meet the rule criteria. The modeled probability curve results indicate that at 0.8 
mg/cm’, there is no substrate type where the upper prediction bound provides a false positive rate 
of <10%. At 1.2 mg/cm”, no false negative rates <5% were obtained based on the lower 
prediction bound. 


Based on the modeled probabilities, a false negative threshold value (the lead level, with 95% 
confidence, above which the test kit would yield fewer than 5% false negative results) of 1.40 
mg/cm? could be established for the LeadPaintCheck test kit. Based on these same probabilities, 
an overall false positive threshold value (i.e., the lead level, with 95% confidence, below which 
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the test kit would yield fewer than 10% false positive results) of 0.24 mg/cm? lead could be 
established. 


After controlling for the significant covariates, the likelihood of a positive test result is positively 
and significantly associated with: higher lead levels and metal and plaster substrates. It is not 
significantly and positively associated with drywall and wood substrates. 


Both the technical and non-technical operator found the LeadPaintCheck test kit instructions to 
be clear, informative, and easy to follow. Step-by-step instructions were provided in the manual 
with color pictures of each step being completed. An instructional DVD was provided with the 
instructions. The solutions used for different steps were easily identifiable within the kit and the 
storage conditions of the reagents were readily marked. All reagents came prepared and ready to 
use. 


The LeadPaintCheck test kit as supplied for this verification test came with an eXact® 
LEADQuick™ Photometer, all necessary reagents (including the quality check standard and the 
photometer strips), a homogenizer kit by Omni that included all components necessary to operate 
the homogenizer, a 500 mL wash/squirt bottle, 15 mL plastic conical tubes with lids, 50 mL 
plastic conical tubes with lids, ¥2 inch cork bore, cork bore sharpener, 100 uL pipette and 
associated disposable tips, % inch cork bore (for applications not part of this verification test), 
scissors, forceps, rod and base to hold the homogenizer, small brush, funnel system for paint chip 
collection, scalpel with blades, masking tape, and a cork bore remover. DI water was not 
supplied with the test kit. The homogenizer came in its own case. All of these components 
approximate the LeadPaintCheck test kit startup kit, which supplies everything listed here in 
sufficient quantities for 50 tests. 


The HP sample solution, diluted HP sample solution, rinses for the photometer, eXact® Strip Pb- 
3 strips, and pipette tips were produced as waste for a single test. The test kit instructions 
indicated that the 15 and 50 mL conical tubes could be re-used. If a positive result is obtained 
with the test kit, most of these components would be considered to contain lead. The eXact® 
Strip Pb-3 and pipette tips should be thrown away in the regular trash. According to the test kit 
instructions, the HP solutions (regular and diluted) should be flushed down the drain with 
copious amounts of water and the conical tubes should then be rinsed and allowed to dry before 
re-use. (Note: Because regulations for the disposal of wastes generated from the use of lead test 
kits may vary from state to state, EPA recommends that test kit users contact their state 
government agency for proper waste disposal requirements.) Rinses used for the photometer 
should be discarded into a sink and flushed with water. 


The LeadPaintCheck test kit provided clear and user-friendly instructions, but required multiple 
steps. Operation of the test kit took approximately 22 minutes for the technical operator and 
approximately 27 minutes for the non-technical operator, not including clean-up time. The non- 
technical operator considered the LeadPaintCheck test kit to be inconvenient for use. A regular 
power supply was needed for the operation of the homogenizer. Four AAA batteries were 
needed for the operation of the eXact® LEADQuick™ Photometer. The LeadPaintCheck Startup 
Kit is priced at $1999.99. The reagent replacement set is $99.99 for 50 tests. After the initial 
Startup Kit investment the cost per paint test is $2. 
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Appendix A 


Performance Evaluation Materials 
Summary Information 


List of Abbreviations and Acronyms 


AMS Advanced Monitoring Systems 

ASTM American Society for Testing and Materials 
CCV continuing calibration verification 

CoV coefficient of variation 

CRM certified reference material 

DI deionized 

EPA U.S. Environmental Protection Agency 

ESTE Environmental and Sustainable Technology Evaluations 
ETV Environmental Technology Verification 

FT film thickness 

HVLP high volume/low pressure 

ICP-AES inductively coupled plasma-atomic emission spectrometry 
ICS interference check sample 

ICV initial calibration verification 

LCS laboratory control spike 

ug/L micrograms per liter 

uL microliters 

mg/cm? milligrams per centimeter squared 

mg/kg milligrams per kilogram 

mL milliliter 

MSDS material safety data sheets 

NLLAP National Lead Laboratory Accreditation Program 
PEM performance evaluation material 

ppb parts per billion 

QA quality assurance 

QC quality control 

QMP quality management plan 

RH relative humidity 

RPD relative percent difference 

SOP standard operating procedure 


Section Al 


Preparation of 
Performance Evaluation Materials 


A-3 


Executive Summary 


Battelle prepared a batch of performance evaluation materials (PEMs) for use in an 
Environmental Technology Verification (ETV) program evaluation of the performance of lead 
paint test kits. These PEMs encompass two lead types (white lead [lead carbonate] and yellow 
lead [lead chromate]), four separate substrates (metal, wood, drywall, and plaster), and six lead 
levels within each lead type (0.3, 0.6, 1.0, 1.4, 2.0, and 6.0 mg/cm’). The goal of the production 
was to produce panels at a specified lead level with minimal variability across and within panels. 
The study design called for a verification and homogeneity study involving inductively coupled 
plasma (ICP) testing of the painted metal panels to determine applied lead levels. Initial 
application procedures included spray application for paints at 2.0 and 6.0 mg/cm’, but testing 
indicated that spray application yielded high variability in lead levels. As a result, the Battelle 
team, in consultation with U.S. Environmental Protection Agency (EPA), decided to apply all 
lead paint layers via drawdown bar, which enables more precision in the thickness of the paint 
layer applied. Later in the development process, continued high variability measurements led to 
the team’s decision to include silica in the formulations of each lead paint to thicken the paint 
and allow for a more even coating. 


Verification and homogeneity testing was conducted for all 12 lead paints as well as the one no- 
lead control paint. Verification testing determined the formulation and drawdown bar best suited 
to yield a particular lead level. Homogeneity test results were assessed for proximity to target 
lead levels, lead level range, and variability within and between panels. All paints passed 
verification and homogeneity testing. 


After completing the verification and homogeneity testing, base paint layers were applied for all 
12 sets of lead paints (two lead types by six lead levels) and the no-lead paint. Paint chips were 
sampled and analyzed from the metal reference panels within each set of PEMs. The metal 
reference panel measurements met target specifications for all sets of PEMs. All nine sets of 468 
panels each were appropriately labeled and packaged. All reference PEM concentrations and 
homogeneity results were reviewed and approved by EPA prior to full-scale production of a set. 


Study Design 


The initial study design specified production of the ETV PEMs using six lead levels (0.3, 0.6, 
1.0, 1.4, 2.0, and 6.0 mg/cm”), two lead types (white and yellow lead), four substrates (wood, 
metal, drywall, and plaster), and three topcoat colors (white, red-orange, and grey-black), as 
specified in Table A-1. For the wood substrates, both poplar and pine wood panels were 
produced, segregated, and uniquely labeled to be consistent with the design in Table A-1. 


The final design specified production of 624 panels for each of seven test kits for a total of 4,368 
panels. Late in the development process, the planned evaluation design changed so that only 468 
panels were required to test each of nine test kits for a total of 4,212 PEMs needed for the ETV 
test. 


Table A-1: PEMs Produced for ETV Evaluation 




































































































































































































# Samples Produced Per Test Kit by Topcoat Color 
Lead Level 
Lead Type (mg/cm?) Substrate White Red-Orange | Grey-Black Total 
Wood-Poplar 2 2 2 6 
Wood-Pine 2 2 2 6 
Control Blank 0 Metal 4 4 4 12 
Drywall 4 4 4 12 
Plaster 4 4 4 12 
Wood-Poplar 2 2 2 6 
Wood-Pine 2 2 2 6 
0.3 Metal 4 4 4 12 
Drywall 4 4 4 12 
Plaster 4 4 4 12 
Wood-Poplar 2 2 2 6 
Wood-Pine 2 2 2 6 
0.6 Metal 4 4 4 12 
Drywall 4 4 4 12 
Plaster 4 4 4 12 
Wood-Poplar 2 2 2 6 
Wood-Pine 2 2 2 6 
1.0 Metal 4 4 4 12 
Drywall 4 4 4 12 
White Lead Plaster 4 4 4 12 
(ead Wood-Poplar 2 2 2 6 
Carbonate) 
Wood-Pine 2 2 2 6 
1.4 Metal 4 4 4 12 
Drywall 4 4 4 12 
Plaster 4 4 4 12 
Wood-Poplar 2 2 2 6 
Wood-Pine 2 2 2 6 
2.0 Metal 4 4 4 12 
Drywall 4 4 4 12 
Plaster 4 4 4 12 
Wood-Poplar 2 2 2 6 
Wood-Pine 2 2 2 6 
6.0 Metal 4 4 4 12 
Drywall 4 4 4 12 
Plaster 4 4 4 12 
Wood-Poplar 36 
Yellow Lead | 95 96 10 Wood-Pine | 2 panels per cell for Wood substrates, 4 36 
(Lead 1. 4 20 6.0 Metal panels per cell for other substrates (same 72 
Chromate) anon Drywall design as White Lead panels) 72 
Plaster 72 
| | Subtotal-PerTestkit _—'||_ 208 | 208 |_—a0s||aa 


The original design plan called for a target lead concentration of 0.3 mg/cm” for a set of PEMs. 
During the writing of the ETV test/quality assurance plan, preliminary ICP results indicated that 
the target level for this set of PEMs might be closer to 0.4 mg/cm’. The preliminary results were 
used in the ETV test plan. 
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Substrate Preparation 


The ETV PEMs included four different substrate types — metal, wood, drywall, and plaster; 
although two types of wood (pine and poplar) were utilized. The following bulleted lists 
describe the steps taken to prepare each of the types of substrates. 


Metal 
¢ Iron Phosphate Steel panels 0.032” x 3” x 3” were placed in an isopropyl alcohol bath and 
carefully wiped and dried before being placed in plastic bags prior to coating. 
¢ The solvent wipe step was performed to ensure that residue oils/fingerprints from the 
manufacturing processes were removed. 


Wood 
¢ Wood (pine and poplar) was purchased in 4” widths, planed, and cut into 3” x 3” panels. 
¢ PEMs were placed into constant temperature and humidity conditioning rooms prior to 
coating application to ensure uniform water content through each panel prior to coating. 
[Note that plaster and drywall panels are less sensitive to water absorption prior to 
coating. | 


Drywall 
° 4" x 8” x 3/8” gypsum drywall sheets were cut into 3” x 3” panels. 


Plaster 
* Two joint compound materials were evaluated for ease of application and smoothness to 
ensure the best surface for coating. USG Joint Compound provided the smoothest 
surface and was used to coat panels at about 1/32” thickness. 
° A3"x4' strip of 3/8” thick gypsum drywall was placed into jig, then plaster joint 
compound was smoothed over top surface to a precise 1/32” thickness. Plastered drywall 
strips were then cut down into 3” x 3” panels. 


Sealer Application to Drywall and Plaster PEMs 
¢ Stacks of drywall and plaster PEMs were sealed on cut edges with no lead latex 
primer/sealer to eliminate dusting. 


All panels were then placed in constant temperature and humidity conditioning rooms prior to 
coating application. 


Spray Application Facilities and Equipment 


Battelle’s laboratory includes a walk-in spray booth capable of this type of production as well as 
air handling equipment and monitors to ensure the safety of Battelle staff. Although the 0.3 and 
6.0 mg/cm’ white lead and no-lead paints were applied via spray application, all other 
application of lead paint layers was performed using drawdown bars in a laboratory setting. All 
topcoats were applied by spray application in the spray booth. Details on the equipment used in 
these processes are listed below. 


Spray Booth 
¢ 10'x 10’x 7.5’ double door spray booth 
¢ Compressed air supply for spray equipment 
¢ Spray equipment consists of a high volume/low pressure (HVLP) gravity fed DeVilbiss 
spray gun 
¢ Plastic sheeting covering walls and floor to minimize clean-up time 


Conditioning Rooms 

¢ Constant temperature (75°Farenheit)) and humidity (50% relative humidity [RH])) rooms 
for substrate conditioning (the variability in temperature and RH is not tracked in those 
rooms) 

¢ Substrates were conditioned both before and after coating application. Wood substrates 
were conditioned a minimum of two weeks prior to coating. All substrates were 
conditioned a minimum of 48 hours after coating and before bagging and wrapping. 

¢ Plastic covering was placed on the floor to minimize clean-up time after transporting 
drying racks from the coating application lab into the conditioning rooms. 


Environmental Health and Safety 


Battelle developed a health and safety plan related to producing lead-based paint and PEMs 
coated with these paints. The plan was approved internally by appropriate environmental safety 
and health personnel. Environmental monitoring during paint mixing and spraying activities 
determined that lead exposure levels for workers were below Occupational Safety and Health 
Administration standards. Some of the components of the safety plan included: 

e All staff and any visitors were required to have documented hazard communication 
training on lead. 

¢ Baseline and post-work blood-lead levels were obtained for those Battelle staff that 
conducted the paint mixing and spray painting. 

¢ Respirators were used during leaded paint production 

¢ Spray application operations staff were required to have a physical, appropriate training, 
and to pass a respirator fit test. 

e The interior of the spray booth was covered with plastic or other material that could be 
easily removed and was then disposed of as hazardous waste. 

e The area in front of the booth was set up as a change-out area where personal protective 
equipment, such as coveralls, etc., could be removed without spreading lead outside of 
the area. 

e Warning signs restricting access were posted at the paint booth door. 


Preparation of Linseed Oil Based Leaded Paints 


To formulate historically accurate lead-based paints to apply to PEMs, Battelle consulted 
Bennett’s The Chemical Formulary — A Collection of Valuable, Timely, Practical Commercial 
Formulae and Recipes for Making Thousands of Products in Many Fields of Industry, Volume 
VI.' The Chemical Formulary had been printed with revisions every year until at least 1998. 
Sample formulations from this reference are listed below in Table A-2. Since the paints 
produced for the ETV verification of lead test kits were being applied to metal, drywall, plaster 
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and wood, Battelle used a combination of formulations from Chapter Thirteen — Paint, Varnish, 
Lacquer and Other Coatings to ensure adhesion to all substrates. Battelle reviewed the various 
relevant historical formulations and developed formulations to apply to the PEMs that would 
work best for application to the four substrates being used, i.e. would provide the best adhesion 
to the variety of substrates required while achieving desired target lead levels. 


Table A-2: Paint Formulations from The Chemical Formulary 


Floor Painting and Finishing (p. | Plaster, Primer (p. 332) Exterior House Paint Pigments 
281) (for raw wood) White (p. 328) 


Soft Paste White Lead, 100 lb. White Lead, Semi-Paste, 100 1b. |35% Leaded Oxide, 45 Ib. 


Raw Linseed Oil, 3 gal. Interior Varnish, 4 gal. White Lead, 18 Ib. 

Turpentine, 2 gal. Linseed Oil, Kettle Bodied, 2 gal. | Titanium Dioxide, 15 Ib. 

Liquid Drier, 1 pt. Turpentine, % gal. Inert, 22 lb. (Battelle used Zinc 
Oxide) 





In preparing the lead-based paints for the PEMs, Battelle used a combination of raw and boiled 
linseed oil to ensure realistic drying time and good adhesion to the variety of substrates. A 
variety of other formulas in the reference also mix these two resins. 


A similar formulation was also found in Charles Uebele’s Paint Making and Color Grinding: A 
Practical Treatise for Paint Manufacturers and Factory Managers’. The excerpt below explains 
the difference in formulation requirements based on the substrate to which the paint will be 
applied. 


“CHAPTER XXV - DIPPING PAINTS. 

Dipping Paints for Wood or Metal require to be made specially for either surface, as that 
intended for wood will not always serve the purpose for metal. The paint for wood 
requires to contain a pigment that acts as a filler, while tin or smooth sheet iron or steel 
does not necessarily need it, in fact, it is best without it for certain metallic surfaces. The 
function of a dipping paint is, first of all, to economize in labor, to cover uniformly 

any article immersed in it, and to dip freely without leaving fringes of paint at the edges 
and dry equally all over the surface thus coated. 


Metal Preservative Red may be made by grinding a base of 40 pounds bright red oxide of 
95 per cent, purity, 8 pounds red lead, 2 pounds zinc chromate, 25 pounds floated silex or 
silica in 25 pounds raw linseed oil thinning same with 5 gallons raw linseed oil, 1 gallon 
hard gum japan and Y% gallon turps. This will produce 12 gallons of paint weighing a 
trifle over Il pounds per gallon. By substituting a long stock of hard gum varnish for part 
of the 5 gallons raw oil a hard drying product will be the result.” 


In support of achieving consistent application of the lead-based paints in terms of film thickness 
and lead level, Battelle investigated additions of various elements to mitigate settling and 
improve application. Silicon dioxide was selected for this purpose because it was present in pre- 
1978 leaded paints, is used for thickening and anti-settling properties in modern paint 
formulations, and achieved the most consistent results. Battelle established the historical 
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precedent for including silica in paints in a technical report submitted to EPA on February 19, 
2009”. 


The primer and topcoats applied to the PEMs on top of the lead-based paints (or base paint for 
the no-lead panels) all contain some form of Diatomaceous Silica, as well. The primer and three 
topcoats applied are listed below. 

e Sherwin Williams brand PreRite Bonding Primer 

e Sherwin Williams Classic 99 Interior Satin Latex color Extra White 

e Sherwin Williams Classic 99 Interior Satin Latex color 7047 Software (Grey) 

e Sherwin Williams Accents Interior Satin Latex color 6867 Fireworks (red-orange) 


Section 2 of the primer Material Safety Data Sheet (MSDS) specifies that the primer contains 9% 
quartz. Quartz is referred to as “Crystalline Silica” in Section 11 of the MSDS. The MSDSs for 
the three topcoats specify Cristobalite (CAS 14464-46-1) as an ingredient, which is a synonym 
for silicon dioxide and also referred to as Crystalline Silica in Section 11. All panels have some 
level of silica in the topcoat layers. 


The paint formulations used for this effort were based on historical records. Primary ingredients 
included zinc oxide, raw and boiled linseed oil, turpentine, Japan drier, either lead carbonate or 
lead chromate, and titanium dioxide (used to balance the levels of lead). Nine different paint 
formulations were produced as dictated by the two lead pigments (lead carbonate and lead 
chromate) and the six different lead levels in addition to the zero lead level control. The 
formulations were designed to consistently achieve the lead levels required when applied at 
typical wet film builds. 


The paint formulations are shown in Tables A-3 and A-4 below. Since the molecular 
compositions of the two lead pigments are different, the formulations have accounted for these 
differences by adjusting the load levels. However, the formulations for the 0% lead chromate 
and carbonate were the same because no lead pigment was used in either. 


Table A-3. White Lead (Lead Carbonate) Paint Formulations 


0% Lead Carbonate Paint Formulation 
Materials [GW | _Lot# [Supplier [| OLead | OLead %by wt. [Gram wt 






























ZnO 47.3 ZC-X013 The Carry Co. 14.79 59.67% 1491.75 
Pb CO, 51 1401047-267 | American Elements 0.00 0.00% 0.00 
TiO2 37 931407T.12 DuPont 6.16 24.86% 621.56 
Linseed Oil 7.8 83734 Recochem Inc. 1.48 5.97% 149.18 
Boiled Linseed Oil 7.7 83404 Recochem Inc. 0.15 0.60% 14.92 
Turpentine 7.0 83304 Recochem Inc. 2.16 8.70% 217.55 
Japan Drier 7.0 PJD 40 Barr 0.05 0.20% 5.04 
Total 24.8 100% 2500 























‘Sample reduced to 60% solids, 0% of TS-100 silica added then sprayed to thickness. 


0.3% Lead Carbonate Paint Formulation 
Materials | GW | Lot# [| Supplier__[ 0.3 Lead] 0.3 Lead % by wt. | Gram wt] 


























ZnO 47.3 ZC-X013 The Carry Co. 14.85 59.08% 1477.08 
Pb CO, 51 1401047-267 | American Elements 1.49 5.91% 147.71 
TiO2 37 931407T.12 DuPont 4.95 19.69% 492.36 
Linseed Oil 7.8 83734 Recochem Inc. 1.49 5.91% 147.71 
Boiled Linseed Oil 7 83404 Recochem Inc. 0.15 0.59% 14.77 
Turpentine 7 83304 Recochem Inc. 2.17 8.62% 215.41 
Japan Drier 7 PJD 40 Barr 0.05 0.20% 4.97 
Total 25.1 100% 2500 




















[Sample reduced to 60% solids, 0% of TS-100 silica added then sprayed to 3 mils wet. 


0.6% Lead Carbonate Paint Formulation 
Materials [| GW | Lot# [Supplier __[ 0.6 Lead] 0.6 Lead % by wt. | Gram wt| 












































ZnO 47.3 ZC-X013 The Carry Co. 14.77 58.45% 1461.22 
Pb CO; 51 1401047-267 | American Elements 1.77 7.00% 175.11 
TiO2 37 931407T.12 DuPont 4.92 19.47% 486.74 
Linseed Oil 7.8 83734 Recochem Inc. 1.48 5.86% 146.42 
Boiled Linseed Oil 7.7 83404 Recochem Inc. 0.15 0.59% 14.84 
Turpentine z 83304 Recochem Inc. 2.15 8.51% 212.70 
Japan Drier 7 PJD 40 Barr 0.03 0.12% 2.97 
Total 25.3 100% 2500 


‘Sample reduced to 70% solids, 0.7% of TS-100 silica added then drawdown with # 24 bar. 


1.0% Lead Carbonate Paint Formulation 





























Materials | Gw | Lot# Supplier 1.0 Lead| 1.0 Lead %by wt | Gram wt 

ZnO 47.3 ZC-X013 The Carry Co. 14.40 55.53% 832.88 

Pb CO, 51 1401047-267 | American Elements 3.00 11.57% 173.52 

TiO2 37 931407T.12 DuPont 4.80 18.51% 277.63 
Linseed Oil 7.8 83734 Recochem Inc. 1.44 5.55% 83.29 
Boiled Linseed Oil 7.7 83404 Recochem Inc. 0.14 0.56% 8.33 

Turpentine 7 83304 Recochem Inc. 2.10 8.10% 121.46 
Japan Drier 7 PJD 40 Barr 0.05 0.19% 2.89 
Total 25.9 100% 1500 

















This formulation will be used to produce 0.6% and 1.4% lead levels at different coating thickness. 





Materials | GW | Lot | Supplier |1.4Lead] 1.4Lead %by wi [Gram wil 


























ZnO 47.3 ZC-X013 The Carry Co. 13.22 50.94% 764.16 

Pb CO; 51 1401047-267 | American Elements 4.21 16.22% 243.35 

TiO2 37 931407T.12 DuPont 4.81 18.54% 278.03 
Linseed Oil 7.8 83734 Recochem Inc. 1.44 5.55% 83.24 
Boiled Linseed Oil 7.7 83404 Recochem Inc. 0.14 0.54% 8.09 

Turpentine 7 83304 Recochem Inc. 2.1 8.09% 121.39 
Japan Drier 7 PJD 40 Barr 0.03 0.12% 1.73 
Total 26.0 100% 1500 












































ZnO 47.3 ZC-X013 The Carry Co. 12.88 48.16% 722.42 

Pb CO; 51 1401047-267 | American Elements 6.08 22.73% 340.98 

TiO2 37 931407T.12 DuPont 4.12 15.41% 231.17 
Linseed Oil 7.8 83734 Recochem Inc. 1.41 5.28% 79.20 
Boiled Linseed Oil 7.7 83404 Recochem Inc. 0.14 0.53% 7.92 

Turpentine 7 83304 Recochem Inc. 2.06 7.70% 115.50 
Japan Drier 7 PJD 40 Barr 0.05 0.19% 2.80 
Total 26.7 100% 1500 









































ZnO 47.3 ZC-X013 The Carry Co. 4.70 16.73% 250.89 

Pb CO, 51 1401047-267| American Elements| 18.10 64.49% 967.34 

TiO2 37 931407T.12 DuPont 1.57 5.58% 83.63 
Linseed Oil 7.8 83734 Recochem Inc. 1.43 5.09% 76.40 
Boiled Linseed Oil 7 83404 Recochem Inc. 0.14 0.51% 7.64 

Turpentine 7 83304 Recochem Inc. 2.09 7.43% 111.42 
Japan Drier 7 PJD 40 Barr 0.05 0.18% 2.67 
Total 28.1 100% 1500 





























‘Sample reduced to 70% solids, 1% of TS-100 silica added then sprayed to thickness. 
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0% Lead Carbonate Paint Formulation 
































‘Sample reduced to 

















Materials _[ GW | Low | Supplier OLead | OLead %bywt_| Gram wi] 
ZnO 47.3 ZC-X013 The Carry Co. 14.79 59.67% 1491.75 
Pb CO; 51 1401047-267 | American Elements 0.00 0.00% 0.00 
TiO2 37 931407T.12 DuPont 6.16 24.86% 621.56 
Linseed Oil 7.8 83734 Recochem Inc. 1.48 5.97% 149.18 
Boiled Linseed Oil 7.7 83404 Recochem Inc. 0.15 0.60% 14.92 
Turpentine 7.0 83304 Recochem Inc. 2.16 8.70% 217.55 
Japan Drier 7.0 PJD 40 Barr 0.05 0.20% 5.04 
Total 24.8 100% 2500 
60% solids, 0% of TS-100 silica added then sprayed to thickness. 















































0.3% Lead Carbonate Paint Formulation 
Watevals [GW [Low | [03 Lead %by wt [Gram wi] 
ZnO 47.3 ZC-X013 The Carry Co. 14.85 59.08% 1477.08 
Pb CO; 51 1401047-267| American Elements 1.49 5.91% 147.71 
TiO2 37 931407T.12 DuPont 4.95 19.69% 492.36 
Linseed Oil 7.8 83734 Recochem Inc. 1.49 5.91% 147.71 
Boiled Linseed Oil 7.7 83404 Recochem Inc. 0.15 0.59% 14.77 
Turpentine ie 83304 Recochem Inc. 247 8.62% 215.41 
Japan Drier 7 PJD 40 Barr 0.05 0.20% 4.97 
Total 25.1 100% 2500 







































































This formulation wi 














Materials | GW | low | Supplier _[0.6Lead] 0.6Lead %by wi [Gram wi| 
ZnO 47.3 ZC-X013 The Carry Co. 14.77 58.45% 1461.22 
Pb CO; 51 1401047-267| American Elements 1.77 7.00% 175.11 
TiO02 37 931407T.12 DuPont 4.92 19.47% 486.74 
Linseed Oil 7.8 83734 Recochem Inc. 1.48 5.86% 146.42 
Boiled Linseed Oil 7.7 83404 Recochem Inc. 0.15 0.59% 14.84 
Turpentine iG 83304 Recochem Inc. 2.15 8.51% 212.70 
Japan Drier 7 PJD 40 Barr 0.03 0.12% 2.97 
Total 25.3 100% 2500 
Sample reduced to 70% solids, 0.7% of TS-100 silica added then drawdown with # 24 bar. 
1.0% Lead Carbonate Paint Formulation 
Materials [| cw [| Lott | Supplier 1.0 Lead] 1.0 Lead %by wt | Gram wt 
ZnO 47.3 ZC-X013 The Carry Co. 14.40 55.53% 832.88 
Pb CO3 51 1401047-267| American Elements 3.00 11.57% 173.52 
TiO2 37 931407T.12 DuPont 4.80 18.51% 277.63 
Linseed Oil 7.8 83734 Recochem Inc. 1.44 5.55% 83.29 
Boiled Linseed Oil 7.7 83404 Recochem Inc. 0.14 0.56% 8.33 
Turpentine 4 83304 Recochem Inc. 2.10 8.10% 121.46 
Japan Drier 7 PJD 40 Barr 0.05 0.19% 2.89 
Total 25.9 100% 1500 








Il be used to produce 0.6% and 1.4% lead levels at different coating thickness. 








1.4% Lead Carbonate Paint Formulation 












































Materials_[ GW | Low | Supplier [1.4 Lead] 1.4 Lead %by wi | Gram wi| 
ZnO 47.3 ZC-X013 The Carry Co. 13.22 50.94% 764.16 
Pb CO, 51 1401047-267| American Elements 4.21 16.22% 243.35 
Tio2 37 931407T.12 DuPont 4.81 18.54% 278.03 
Linseed Oil 7.8 83734 Recochem Inc. 1.44 5.55% 83.24 
Boiled Linseed Oil 7.7 83404 Recochem Inc. 0.14 0.54% 8.09 
Turpentine 7 83304 Recochem Inc. 2.1 8.09% 121.39 
Japan Drier 7 PJD 40 Barr 0.03 0.12% 1.73 
Total 26.0 100% 1500 
‘Sample reduced to 70 % solids, 1.5% of TS-100 silica added then drawdown with # 54 bar. 


bonate Paint Formulation 















































Materials[ GW | low | Supplier | 20Lead] 20 Lead “by wt | Gram wi| 
ZnO 47.3 ZC-X013 The Carry Co. 12.88 48.16% 722.42 
Pb CO; 51 1401047-267| American Elements 6.08 22.73% 340.98 
TiO2 37 931407T.12 DuPont 4.12 15.41% 231.17 
Linseed Oil 7.8 83734 Recochem Inc. 1.41 5.28% 79.20 
Boiled Linseed Oil 7.7 83404 Recochem Inc. 0.14 0.53% 7.92 
Turpentine 7 83304 Recochem Inc. 2.06 7.70% 115.50 
Japan Drier 7 PJD 40 Barr 0.05 0.19% 2.80 
Total 26.7 100% 1500 
Sample reduced to 65% solids, 1.5% of TS-100 silica added then drawdown with #40 bar. 


bonate Paint Formulation 
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Wateriais[ GW | low | Supplier [60Lead] 60 Lead “by wt | Gram wi| 

ZnO 47.3 ZC-X013 The Carry Co. 4.70 16.73% 250.89 

Pb CO, 51 1401047-267| American Elements | 18.10 64.49% 967.34 
TiO2 37 931407T.12 DuPont 1.57 5.58% 83.63 
Linseed Oil 7.8 83734 Recochem Inc. 1.43 5.09% 76.40 
Boiled Linseed Oil 77 83404 Recochem Inc. 0.14 0.51% 7.64 

Turpentine 7 83304 Recochem Inc. 2.09 7.43% 111.42 
Japan Drier 7 PJD 40 Barr 0.05 0.18% 2.67 
Total 28.1 100% 1500 


Sample reduced to 70% solids, 1% of TS-100 silica added then sprayed to thickness. 








Table A-4. Yellow Lead (Lead Chromate) Paint Formulations 


0.3% Lead Chromate Paint Formulation 
Materials] GW | Lot | Supplier [0.3Lead] 0.3Lead %by wt | Gram wi 
ZnO ZC-X013 The Carry Co. 
PbCrO, 1401047-267| American Elements 
Ti02 931407T.12 DuPont 

Linseed Oil f 83734 Recochem Inc. 
Boiled Linseed Oil 83404 Recochem Inc. 0.60% 15.01 

Turpentine 83304 Recochem Inc. 8.75% 218.86 
Japan Drier PJD 40 Barr 0.20% 5.01 


100% 2500 


















































0.6 Lead] 0.6 Lead % by wt. 

ZC-X013 The Carry Co. 57.52% 1437.97 
1401047-267| American Elements 8.44% 211.03 
931407T.12 DuPont 19.16% 478.99 
Linseed Oil 83734 Recochem Inc. 5.77% 144.29 
Boiled Linseed Oil 5 83404 Recochem Inc. 0.59% 14.72 

Turpentine 83304 Recochem Inc. 8.40% 210.05 


Japan Drier PJD 40 Barr 0.12% 2.94 
100% 2500 


Sample reduced to 70 % solids, 1.5% of TS-100 silica added then drawdown with #24 bar. 




































1.0% Lead Chromate Paint Formulation 


Materials | Gw | Low | Supplier 1.0 Lead| 1.0 Lead %by wt | Gram wt!| 
ZnO 47.3 ZC-X013 The Carry Co. 14.40 55.53% 832.88 














PbCrO, 
Ti02 37 931407T.12 DuPont 
Linseed Oil 7.8 83734 Recochem Inc. 
Boiled Linseed Oil : 83404 Recochem Inc. 
Turpentine 83304 Recochem Inc. 
Japan Drier PJD 40 Barr 




















This formulation will be used to produce 0.6% and 1.4% lead levels at different coating thickness. 
Sample reduced to 70 % solids, 1% of TS-100 silica added then drawdown with #48 bar. 



































Materials 
ZnO ZC-X013 The Carry Co. 
PbCrO, 1401047-267 | American Elements 
Ti02 931407T.12 DuPont 
Linseed Oil - 83734 Recochem Inc. 
Boiled Linseed Oil : 83404 Recochem Inc. 
Turpentine 83304 Recochem Inc. 
Japan Drier PJD 40 Barr 




















Materials 
ZnO ZC-X013 The Carry Co. 
PbCrO, 1401047-267| American Elements 

Ti02 931407T.12 DuPont 

Linseed Oil 83734 Recochem Inc. 
Boiled Linseed Oil 83404 Recochem Inc. 14.50 

Turpentine 83304 Recochem Inc. 211.46 
Japan Drier PJD 40 Barr 4.85 
































Materials 

ZnO ZC-X013 The Carry Co. 5.99% 149.69 
PbCrO, 1401047-267| American Elements 77.84% 1946.00 
Ti02 931407T.12 DuPont 2.00% 49.90 
Linseed Oil 83734 Recochem Inc. 5.47% 136.75 
Boiled Linseed Oil : 83404 Recochem Inc. 0.55% 13.68 
Turpentine 83304 Recochem Inc. 7.98% 199.43 

Japan Drier PJD 40 Barr 0.18% 4.54 

100% 














Sample reduced to 70% solids, 2% of TS-100 silica added then drawdown with #54 bar. 
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0.3% Lead Chromaite Paint Formulation 














Materials | GW | Lot# | Supplier__| 0.3 Lead] 0.3 Lead % by wt. | Gram wt} 
ZnO ZC-X013 The Carry Co. 14.97 60.03% 1500.74 
PbCrO, 1401047-267 | American Elements 1.10 4.40% 
Ti02 931407T.12 DuPont 
Linseed Oil 83734 Recochem Inc. 
Boiled Linseed Oil 83404 Recochem Inc. 
Turpentine 83304 Recochem Inc. 
Japan Drier 
































Materials 
ZnO ZC-X013 The Carry Co. 57.52% 1437.97 
PbCrO, 1401047-267 | American Elements 8.44% 211.03 
Ti02 931407T.12 DuPont 19.16% 478.99 
Linseed Oil Recochem Inc. 
Boiled Linseed Oil Recochem Inc. 
Turpentine Recochem Inc. 
Japan Drier Barr 











1.0% Lead Chromate Paint Formulation 





Sample reduced to 70 % solids, 1.5% of TS-100 silica added then drawdown with #24 bar. 

































Materials | GW | Lot# | Supplier__| 1-0 Lead| 1.0 Lead % by wt | Gram wt} 

ZnO a3 
PbCrO, 51 1401047-267 | American Elements 11.57% 
Ti02 37 931407T.12 DuPont 18.51% 
Linseed Oil 7.8 83734 Recochem Inc. 5.55% 

Boiled Linseed Oil rea Recochem Inc. 8.33 
Turpentine 7 Recochem Inc. 121.46 
Japan Drier "4 Barr 2.89 

Total 100% 


This formulation will be used to produce 0.6% and 1.4% lead levels at different coating thickness. 









Materials 





1.4% Lead Chromate Paint Formulation 
| GW | Lott | Supplier | 1.4 Lead| 1.4 Lead %by wt | Gram wt| 


Sample reduced to 70 %solids, 1% of TS-100 silica added then drawdown with #48 bar. 







































ZnO 


ZC-X013 





The Carry Co. 





ZnO 473 
PbCrO, 1401047-267 | American Elements 19.42% 
Ti02 931407T.12 DuPont 16.02% 
Linseed Oil 83734 Recochem Inc. 5.49% 
Boiled Linseed Oil Recochem Inc. 
Turpentine Recochem Inc. 
Japan Drier Barr 





42.32% 1058.12 


























Turpentine 7 


Recochem Inc. 


PbCrO, 51 1401047-267 | American Elements 27.83% 695.72 

Ti02 37 931407T.12 DuPont 14.81% 370.34 
Linseed Oil 7.8 83734 Recochem Inc. 5.80% 145.00 
Boiled Linseed Oil 83404 Recochem Inc. 0.58% 14.50 












211.46 





Japan Drier rh 





Barr 








4.85 

















ZC-X013 





The Carry Co. 






1.65 








2500 








5.99% 





149.69 








PbCrO, 51 
Ti02 





1401047-267 
931407T.12 





American Elements 
DuPont 





21.43 









77.84% 1946.00 























Japan Drier 7 


Linseed Oil 7.8 83734 Recochem Inc. 
Boiled Linseed Oil Th 83404 Recochem Inc. 
Turpentine 7 83304 Recochem Inc. 
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Sample reduced to 70% solids, 2% of TS-100 silica added then drawdown with #54 bar. 


Paint Formulation Procedures 


The paint samples were produced using standard painting production procedures in the Battelle 
laboratories, including pre-mixing, media grinding of pigment and binder resin, and paint 
letdown with resin and solvents. This procedure has been used for paint production both in the 
laboratory and in commercial paint manufacturing for over 50 years. The equipment utilized in 
this procedure includes the following: 

e Variac that controls the speed of the dispersator 
High speed dispersator using a 5” diameter blade on the end of the mixing shaft 
Ice bath and ice 
Balance 
Paint cans 
Medium paint filters 
Red Devil paint shaker 


Following are the detailed steps in the paint formulation procedure: 
1. Add enough turpentine to cover mixing blade. 
2. Start mixer at low speed. 
3. Add zinc oxide slowly for 3-5 minutes, increasing mixing speed as needed to maintain 
appropriate grind viscosity as visually evaluated by an operator skilled in the art. 
Add turpentine as needed to keep the batch rolling. 
Mix additional 10 minutes after addition of zinc oxide. 
Add lead pigment slowly for 2-4 minutes, increasing mixing speed as needed. 
Add turpentine as needed to keep the batch rolling. 
Mix additional 10 minutes after addition of lead pigments. 
Add titanium dioxide slowly for 3-5 minutes, increasing mixing speed as needed. 
. Add turpentine as needed to keep the batch rolling. 
. Mix for 60-90 minutes, or until batch viscosity decreases, determined by rolling action of 
the batch. 
. Check Hegman, if < 5 continue to mix, and check Hegman every 10 minutes." 
. When Hegman reaches # or > 5, start the let down, which includes adding all remaining 
liquid raw materials after the pigment and extenders have been dispersed adequately. 
14. Add boiled and raw linseed oil slowly and decrease mixing speed. 
15. Add turpentine to wash out linseed oil container. 
16. Mix additional 10 minutes after addition of linseed oils. 
17. Add Japan drier drop wise to batch. 
18. Mix additional 10 minutes after addition of Japan drier. 
19. Tare quart cans. 
20. Filter batch with medium paint filters into tared quart cans. 
21. Note net weight and log book number of batch on quart cans. 
22. Yields about 1% quarts of lead paint per batch. 
23. Allow paint to set overnight. 
24. Shake paint with Red Devil paint shaker for about 10 minutes take samples for % solids 
check. 
25. Check paint solids with moisture balance and record average of three test results on 
formulation sheet. 
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26. Store paint in aluminum cans in laboratory hood until future use. 


Verification and Homogeneity Studies 


Various batches of paint were prepared for the initial verification tests — one targeting each lead 
level. Each paint was applied via drawdown or hand spraying to 3.5” x 5" metal panels attached 
to a wooden rack. For each paint type and concentration batch, panels were coated to determine 
proper film thickness, formulations, and drawdown bars to use, if applicable, to achieve each 
desired lead level. Subsequently, homogeneity panels were coated to investigate ability to 
achieve target lead levels and variability within and across panels. Verification and homogeneity 
studies were performed on metal panels only due to ease and accuracy of sample extraction, 1.e., 
it was easiest to obtain a 1 inch square sample from the metal surface which led to the most 
accurate measurements of lead content in the sampled area, which was critical for verification 
purposes. 


After drying, paint chip samples were obtained from the metal panels following ASTM E1729.” 
Laboratory analysis for lead by inductively coupled plasma-atomic emission spectrometry (ICP- 
AES) was planned and conducted at an independent National Lead Laboratory Accreditation 
Program (NLLAP)-accredited laboratory, Schneider Laboratories, Inc. ICP-AES testing was 
conducted on three panels for each lead level with four samples obtained from each panel, 
referred to as Homogeneity Panels since the primary purpose of the samples was to assess 
consistency of lead levels across and within panels. The paint chips were digested using EPA 
Method 3050B° and the ICP-AES analysis was conducted following EPA Method 6010C’ as 
well as the Schneider Laboratories, Inc. ICP SOP.* The laboratory electronically reported lead 
level measurements along with quality control (QC) sample results. Laboratory spike and 
duplicate results as well as calibration verification sample results were supplied and reviewed for 
each batch of samples analyzed. Acceptable recoveries for spike samples ranged from 80% to 
120%. Acceptable recoveries for calibration verification samples were 90-110%. Acceptable 
duplicate samples had a relative percent difference of 25% or less. Percent recoveries for 
calibration verification samples ranged from 93-110%. Recoveries for QC spike samples ranged 
from 92-115%. All duplicate samples had less than 25% relative percent difference. There were 
no QC failures or problems. 


Film thickness measurements were obtained by Battelle for each paint sample taken. Results of 
the final batches of homogeneity samples for each set of PEMs are included in Table A-5. 
Results were evaluated to determine correspondence to target lead levels and level of variability 
as measured by the coefficient of variation (CoV), the standard deviation divided by the mean. 
The production plan, agreed to in advance, specified a minimum acceptability of a CoV of less 
than 15 percent. Following analysis, the results were forwarded to EPA with recommendations 
regarding ability to proceed with production or the need for additional homogeneity testing. The 
results shown in Table A-5 met the acceptability requirements and were thus deemed acceptable 
for proceeding with the production of sets of PEMs at each lead level. 


A-15 


Table A-5. Results from Final Homogeneity Testing on Metal Substrates for Each Set of 
ETV PEMs 
















Target Lead Mean Levels CoVv* 
Level__| ICP (mg/cm*) | FT (mils) |__IcP__| 
| os | S| 5] 7.1 









White | 10 | 099] 1.26] 3.9 
Lead 





| 60 | 597] 9414.2 
| os | 30] 1.16] 9.6 
| oe | 2] 0.98] 








bags 
Chromate 






0.1 
5.2 








6.0 
* Coefficient of Variation (Standard Deviation/Mean x 100) 
** Film thickness 





3.3 








Production Application of Lead Paint Coatings 


Based on the results from the Verification and Homogeneity Study summarized in Table A-5, 
production proceeded using the paint formulation and application method (spray or a particular 
size drawdown bar) that achieved the target lead levels. During production application, 
reference panels were coated along with the production panels at a rate of 18 for each set of 468 
panels. For sets of PEMs that were sprayed, reference panels were placed at previously- 
assigned, randomly selected locations on the racks containing all the PEMs awaiting paint 
application. For sets of PEMs that had the lead paint applied via drawdown bar, production 
panels were drawdown in sets of two to three panels each for the wood, metal and drywall 
substrates and one at a time for the plaster substrates. At the discretion of the operator, a 
reference panel was prepared approximately every 10 sets or 25 panels. 


Metal panels were used as the reference panels since metal panels yield the most accurate 
measurements of film thickness and lead levels. The reference PEMs were tested for film 
thickness during application and for lead level by ICP analysis after the paint had dried. This test 
procedure was used to check that the application process resulted in appropriate lead levels. 
Despite the use of the metal substrate only for the reference panels, the lead levels and paint 
thickness on these reference panels served as representative of the coatings applied to all wood, 
drywall, plaster, and metal substrate panels. 


Table A-6 presents the average lead levels, CoV, minimum, and maximum of each set of 18 
reference panel measurements. Most sets are very close to target lead levels, such as the 2.03 
mg/cm’ average for the 2.0 mg/cm” target yellow lead set, the 0.32 mg/cm’ for the 0.3 mg/cm” 
target yellow lead set, and the 0.64 mg/cm’ average for the 0.6 mg/cm” target white lead set. 
There also were a few sets that were a bit off target, but were sufficient to meet the verification 
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needs. Despite the high average lead level of 9.2 mg/cm”, the 6.0 mg/cm? white lead PEMs were 
accepted by EPA because they still met the needs of the verification for a set of PEMs at a high 
lead level. In the 0.6 mg/cm’ yellow lead batch, the measured lead levels of 17 of the 18 
reference panels ranged from 0.51 to 0.66 mg/cm’, yielding a mean of 0.55 mg/cm’, and a CoV 
of 7.5%. Because only one reference panel of 18 yielded a high lead level, the set of panels was 
accepted. 


Table A-6. Reference Panel Results from Final Production for Each Set of ETV PEMs 













Lead Levels Range 
[Mean (mg/cm’)| CoV | Min _| 
| oo) hf 08.2] 0.002 | 
| og | 4] 13.5] 0.425) 


Target Lead 










White 
Lead 0.918 
(lead [14 | '1.48| 8.0] __1.322 
Carbonate) 





| 60 | 9818] 81.2| 5.65 
| oe | 57] 16.6] 0.511] 









Yellow 
Lead 0.879 
(Lead 1.194 
Chromate) 








6.0 


* Next highest measurement was 0.659 


9.6 


5.15 





6.247 








Topcoating 


The linseed oil based paints were applied to the PEMs and stored in the constant temperature and 
humidity rooms during a four to seven day drying time. The panels were then all topcoated with 
Sherwin Williams brand Prep Rite bonding Primer to ensure good adhesion between the linseed 
oil based paint and the latex emulsion topcoat paints. The final latex paint topcoat was then 
applied to the PEMs. The topcoat paints are described in more detail below: 

e Primer — Sherwin Williams Prep Rite bonding primer, diluted with deionized (DI) water 
at a ratio of 3:1 parts by volume. Spray application was done with a 50 percent overlap on 
the PEMs in both horizontal and vertical directions with a total wet film build of 
approximately 4-5 mils (a measure of dry film thickness). The PEMs then were allowed 
1-2 hours to air dry before top coats were applied. 

e Top coat number | is Sherwin Williams Classic 99 interior satin latex; color Extra White, 
diluted with DI water at a ratio of 3:1 parts by volume. Spray application was done with a 
50 percent overlap on the PEMs in both horizontal and vertical directions, with a total 
wet film build of approximately 4-6 mils. Then the PEMs were allowed to air dry for 
three days. The PEMs were then bagged for further testing. 

e Top coat number 2 is Sherwin Williams Classic 99 interior satin latex; color 7047 
(software gray), diluted with DI water at a ratio of 3:1 parts by volume. Spray application 
was done with a 50 percent overlap on the PEMs in both horizontal and vertical 
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directions for a total wet film build of approximately 4-6 mils. Then the PEMs were 
allowed to air dry for three days. The PEMs were then bagged for further testing. 

Top coat number 3 is Sherwin Williams Color Accents interior satin latex; color 6867 
(Fireworks orange red), diluted with DI water at a ratio of 3:1.5 parts by volume. Spray 
application was done with a 50 percent overlap on the samples in both horizontal and 
vertical directions for a total wet film build of approximately 4-6 mils. The PEMs were 
then allowed to air dry for three days. The PEM samples were then bagged for further 
testing. 


PEM Labeling, Packing and Storage 


The PEMs were stored in the constant temperature and humidity conditioning rooms prior to 
being packed up for transfer to the evaluation location. Each PEM was labeled on the back with 
an individual identification number, wrapped in a single laboratory towel to protect the front 
surface, and placed inside an individual zip seal bag also labeled with the identification number. 
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Section A2 


Comparison of Expected vs. Actual Lead Concentrations 
of Performance Evaluation Materials 


The following tables present a comparison of the expected vs. confirmed lead concentration for 
each PEM used during the testing of the lead test kits. Expected concentrations are based on lead 
levels defined for sets of PEMs during the PEM production process. That is, PEMs were being 
made at expected lead concentrations of 0, 0.3, 0.6, 1.0, 1.4, 2.0, or 6.0 mg/cm”. These are the 
expected lead levels as defined in the test/quality assurance (QA) plan. Confirmed 
concentrations are based on ICP-AES results from individual paint chip samples taken from each 
PEM during testing (see Section 3.3.1in the test/QA plan). 


Table A-7 presents the results by substrate and across all PEMs. Table A-8 presents the results 
by lead type. The average and standard deviation for the confirmed lead levels, as well as the 
CoV, are presented for each expected concentration level. 


Table A-7. Confirmed lead level statistics for PEMs compared to expected lead level 
concentrations by substrate type. 
































Expected PEM Lead Level (mg/cm) 
Substrate 0 0.3 0.6 1 1.4 2 6 
N 144 288 282 290 296 288 292 
Drywall Confirmed Lead Level: Average 0.00 0.34 0.83 1.15 1.48 2.52 9.04 
Confirmed Lead Level: StdDev 0.00 0.07 0.22 0.21 0.29 0.33 2.32 
CoV (%) 143.16 20.59 26.01 17.95 19.79 13.22 25.69 
N 144 288 288 288 288 288 286 
Metal Confirmed Lead Level: Average 0.00 0.31 0.56 0.85 1.26 1.91 8.18 
Confirmed Lead Level: StdDev 0.01 0.07 0.14 0.10 0.19 0.28 1.86 
CoV (%) 368.04 22.91 2439 11.31 1491 1449 22.76 
N 140 290 292 296 288 288 284 
Plaster Confirmed Lead Level: Average 0.00 0.44 1.25 1.65 1.79 2.91 10.11 
Confirmed Lead Level: StdDev 0.01 0.18 0.53 0.84 0.65 0.85 3.01 
CoV (%) 258.82 40.17 4240 50.92 36.22 29.24 29.81 
N 144 288 288 275 282 284 288 
Wood Confirmed Lead Level: Average 0.00 0.32 0.72 1.07 1.45 2.39 8.71 
Confirmed Lead Level: StdDev 0.02 0.16 0.22 0.31 0.29 0.71 1.59 
CoV (%) 470.54 48.20 30.83 28.80 = 20.29 -29.56 —:18.29 
N 572 1154 1150 1149 1154 1148 1150 
All Confirmed Lead Level: Average 0.00 0.35 0.84 1.18 1.50 2.43 9.01 
Confirmed Lead Level: StdDev 0.01 0.14 0.41 0.55 0.44 0.69 2.36 
CoV (%) 451.36 38.91 48.34 46.76 29.38 28.49 26.24 





CoV = Coefficient of Variation (Standard Deviation/Mean x 100) 


Table A-7 indicates that overall confirmed lead levels were similar to expected concentrations. 
However, there are substrate types for which, comparatively, the confirmed lead levels were 
higher than the expected levels. Average confirmed levels for drywall and plaster PEMs were 
higher than expected levels, especially when compared to average confirmed lead levels from 
metal and wood. The PEMs used in the verification test were produced mainly using a 
drawdown technique (for all panels except no lead, 0.3 mg/cm” and 0.6 mg/cm’ white lead). 
This involved applying the paint to the PEM and pulling it down with a specially designed bar. 
Being porous substrates, it is possible that the plaster and drywall panels absorbed some of the 
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paint, causing more paint to be applied to the PEM to accommodate the thickness required on the 
PEM. This would then lead to higher lead concentrations on these substrates. The most 
significant potential impact of this effect can be seen on the plaster PEMs. This potential effect 
is based on observations during the production of the PEMs but has not been studied or 
confirmed. 


Table A-8. Confirmed lead level statistics for PEMs compared to expected lead level 
concentrations by lead paint type. 























Expected PEM Lead Level (mg/cm’) 

Lead Type 0 0.3 0.6 1 1.4 2 6 
N 572 
None Confirmed Lead Level: Average 0.00 
Confirmed Lead Level: StdDev 0.01 
CoV (%) 451.36 

N 576 574 573 578 576 572 

White Confirmed Lead Level: Average 0.30 0.88 1.24 1.53 2.36 8.37 

Confirmed Lead Level: StdDev 0.08 0.41 0.72 0.52 0.58 2.05 

CoV (%) 25.56 46.53 58.18 34.08 2460 24.47 

N 578 576 576 576 572 578 

Yellow Confirmed Lead Level: Average 0.40 0.80 1.13 1.46 2.51 9.64 

Confirmed Lead Level: StdDev 0.16 0.40 0.30 0.33 0.78 2.48 

CoV (%) 40.64 49.95 26.32 22.87 31.25 25.75 

N 572 1154 1150 1149 1154 1148 1150 

All Confirmed Lead Level: Average 0.00 0.35 0.84 1.18 1.50 2.43 9.01 

Confirmed Lead Level: StdDev 0.01 0.14 0.41 0.55 0.44 0.69 2.36 

CoV (%) 451.36 38.91 48.34 46.76 29.38 2849 26.24 











CoV = Coefficient of Variation (Standard Deviation/Mean x 100) 


The results in Table A-8 show that there was no significant difference in confirmed lead levels 
between white and yellow lead PEMs. The CoVs values were all <50% at all levels except 0.0 
mg/cm’. The larger CoV at this level is reflective of small changes around the zero lead level 
and most likely represent ICP-AES measurement variability near the detection limit, since no 
lead was used in preparing these PEMs. It should be noted, as discussed in Section A1 of this 
appendix, that the PEMs prepared at the expected lead level of 6.0 mg/cm? were known to be on 
average higher than 6.0 mg/cm” and that it was purposefully decided to accept the variation 
present at this expected lead level. 


Though there were some differences between the confirmed and expected lead levels, it should 
be noted that when evaluated for proper responses, test kit results were compared to confirmed 
lead levels. That is, test kit results were always compared to the actual PEM lead levels, not the 
expected. 
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Section A3 
QA/QC Results for the ICP-AES Analysis of 
Performance Evaluation Materials 
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Summary of Lead Level Confirmation ICP-AES Analysis of PEMs 


All paint chip samples from the PEMs used in this verification test were analyzed using ICP- 
AES by Schneider Laboratories, Inc. 


Sample preparation procedures followed the SOP generated by Schneider Laboratories, Inc. for 
this study (Schneider Laboratories, Inc., SOP Battelle Paint Samples, Doc # III-044-10-001). 
Information on how QC samples were spiked and final concentrations is provided in the SOP. 


Three versions of this SOP (the original and two revisions) were used dated 1/20/10, 2/24/10, 
and 4/25/10. Approximately 27% of the PEMs were analyzed prior to the 2/24/10 revision to the 
SOP. In the 2/24/10 version, revisions were made to clarify that post-digestion matrix spikes and 
duplicates were being evaluated. Additionally, the laboratory control spike (LCS) procedures 
changed such that a separate LCS and a QC check sample were now being performed. 

Originally, in the 1/20/10 version, the LCS was prepared by spiking the QC check sample, which 
was a certified reference material (CRM) (as stated in Section 6.11.2 of the 1/20/10 SOP) 
containing a known quantity of lead. This practice was changed because there were recovery 
issues. The spike concentration of 1000 micrograms per liter (ug/L) was not >3x the background 
lead concentration because of the high lead concentrations in the actual CRM samples (4630 
milligrams per kilogram [mg/kg]). Thus, as of the 2/24/10 SOP, one LCS, one QC check 
sample, and one QC check sample duplicate were being evaluated for every 20 samples. The 
LCS (Blank Paint QC) sample in the 2/24/10 SOP was defined as a piece of non-lead containing 
paint that was spiked with lead solution to a resulting concentration of 1000 ug/L. The QC check 
sample in the 2/24/10 SOP contained 10 mg of the CRM, a known lead-containing material. The 
QC check (CRM) was purchased to contain 4630 + 266 mg/kg lead. To prepare the sample, 10 
mg of the CRM was weighed out and diluted to 10 mL, resulting in a final concentration of 4.630 
mg/L. 


The 4/25/10 revision of the SOP clarified the acceptance criteria for the LCS samples, as it did 
not appear to be clearly defined in previous versions. 


Because of the high lead concentration in the PEM samples, dilutions were made to the samples 
prior to initial analysis. The dilutions were prepared by spiking 10 microliters (uL) of the 
original digested sample into 9.990 milliliters (mL) of reagent water for a 1:1000 dilution. The 
samples were thoroughly mixed by inverting, and then analyzed for lead content. If the result 
was below the reporting limit, the sample was reanalyzed either non-diluted or at a lower dilution 
level. If samples were rerun at a different dilution level, this was noted in the QC summary 
report for that particular sample set. 


The MDL for lead was 2.91 pg/L. 


The reporting limit was 40 pg/L. Therefore all blank results should be <40 pg/L. 
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Summary of Quality Control Measures for PEMs ICP-AES Analysis 


QC procedures were performed in accordance with the quality management plan (QMP) for the 
Battelle ETV Advanced Monitoring Systems (AMS) Center, except where differences were 
noted for Environmental and Sustainable Technology Evaluations (ESTE) per the EPA ETV 
Program QMP, and the test/QA plan for this verification test. Test procedures were conducted as 
stated in the test/QA plan; however a deviation to the test/QA plan was made during the ICP- 
AES analyses. For some sample runs, continuous calibration verification (CCV) samples were 
run once every 20 instead of 10 samples. This deviation is further described below. This change 
was assessed to have no impact on the quality of the results as described below. QC results for 
the analysis of paint chip samples from the PEMs are described below. 


ICP-AES Blank Sample Results 


Various blank samples were analyzed for the ICP-AES analyses. Method blank samples were 
analyzed in each set of 10-20 paint samples to ensure that no sources of contamination were 
present. An initial calibration blank was analyzed at the beginning of each run and used for 
initial calibration and zeroing the instrument. A continuing calibration blank was analyzed after 
each CCV to verify blank response and freedom from carryover. No blank samples failed QC 
during the analyses. 


Calibration Verification Standards 


Initial calibration standards were run at the beginning of each set of analyses. The acceptance 
criterion for the calibration coefficient of the calibration standards was > 0.998. If this criterion 
was not met, the analysis was stopped and recalibration was performed before samples were 
analyzed. A 500 parts per billion (ppb) CCV standard was analyzed at the beginning of each run 
(following the initial calibration), at the end of each run, and every 10-20 samples. CCV 
recoveries ranged from 96% to 108%. Per the test/QA plan, CCV sample frequency was once 
every 10 samples. For most of the sample sets, CC Vs were performed with this frequency. 
However, for later sample sets, CCVs were run once every 20 samples. CCV samples are used 
to verify instrument performance and are evaluated usually at a specified frequency as a 
preventative measure so that large amounts of samples do not need to be re-run if a CCV sample 
fails. In the course of this study, only one CCV sample failed, and it was when the CCV was 
being run once every 10 samples. All samples from the last passing CCV of that sample set were 
re-analyzed. 


QC samples also included an initial calibration verification (ICV) standard and interference 
check sample (ICS). Both samples were 500 ppb. ICV samples were analyzed once at the 
beginning of each sample run and were required to have percent recoveries between 90-110% to 
be acceptable. ICS samples were analyzed at the beginning and end of every run and every 10- 
20 samples. ICS samples had to have percent recoveries between 80-120% to be acceptable. All 
reported ICV and ICS samples met the acceptance criteria. Recoveries for ICV samples ranged 
from 96% to 108%. Recoveries for ICS samples ranged from 93% to 112%. 
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Matrix Spike Samples/Duplicates 


Matrix spike samples, as well as duplicates of these samples, were analyzed once every 10-20 
samples. Acceptable recoveries for matrix spike samples were between 80-120%. Duplicate 
samples had acceptance criteria of +25% relative percent difference (RPD). 


All matrix spike samples were performed as post-digestion spikes as there was insufficient 
sample volume to perform a pre-digestion spike. Matrix spike recoveries ranged from 86% to 
207%. Six matrix spike samples failed with recoveries above the specified acceptance criteria. 
In these instances, the lead concentration in the sample was well above the spike level. Matrix 
spike results indicate that matrix interferences were not observed. Duplicate samples were 
within the specified RPD. 


LCS Samples 


LCS samples were analyzed once every 10-20 samples. Acceptable recoveries for LCS samples 
were between 80-120%. LCS recoveries ranged from 17% to 225%. Schneider Laboratories, 
Inc. noted that LCS failures on one sample set were attributed to improper spiking technique. 
Training on spiking procedures was immediately implemented by Schneider Laboratories, Inc. 
for all analysts spiking samples. All LCS failures occurred prior to a revision to the Schneider 
Laboratories, Inc. SOP for analyzing paint samples for this verification test. In the original 
version of the SOP, LCS samples were prepared by spiking a known amount of lead onto a 
CRM. This practice was changed on 2/24/10 because there were recovery issues. The spike was 
not >3x the background lead concentration because of the high lead concentrations in the actual 
CRM samples. In the revised SOP, the LCS was prepared by spiking a piece of lead-free latex 
paint. There were no LCS failures after that In addition, a QC check sample containing only 
the CRM, which has a known concentration of lead weighed out to a particular amount, was 
analyzed with each sample set throughout the verification test. These QC samples all passed 
acceptance criteria. 
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Appendix B 


Vendor Comments 


Industrial Test Systems, Inc. submitted the following comments on the draft report. These 
comments have not been reviewed by Battelle or U.S. EPA for accuracy, and do not 
necessarily reflect the opinions or views of U.S. EPA. Any questions regarding the 
comments in this section should be addressed to the vendor. 


Industrial Test Systems, Inc. (ITS) reviewed the revised verification report prepared in October 
2010. We have no issues with the report, and we offer our appreciation to those who worked to 
complete the study. 


In the report we learned that our LeadPaintCheck test did not meet the false negative and false 
positive goals as required by the EPA. In the final step of LeadPaintCheck the user compares the 
abs value displayed to limits in the “Interpreting Your abs Result for Lead” table (interpretation 
table) to determine if the sample is positive or negative for lead. Upon review of the test data ITS 
determined that there was an error in the absorbance (abs) values we selected in calibrating our 
interpretation table. 


Our calculations indicate that a small change in this table will allow LeadPaintCheck to meet the 
false negative and false positive goals as required by the EPA. 


We plan to confirm our findings and update our instruction manual to reflect a more accurate 
calibration of the data interpretation table. The technology will not change - the only difference 
for the kit will be the interpretation table. However, to eliminate confusion we plan to rename 
the kit LeadPaintCheck II. It has a new start-up part number 48691 1-2. 


Industrial Test Systems, Inc. 
Rock Hill, S.C 
October 27, 2010 
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